ulogd2 question - meaning of flow.start.sec when hash_mode == 0 (NCFT input, JSON output)
Subsystems (2)
When using hash_mode == 0 with default event_mask, it looks like destruction entry has uninitialized (or having some specific meaning) flow.start.sec and flow.start.usec - as the former converts to early 1970.
Peeking at the code in event_handler_no_hashtable() it looks like it should be set correctly, but alas something seems amiss.
creation event:
{"timestamp": "2019-04-16T12:02:30+0200", "dvc": "testdev", "orig.ip.protocol": 6, "orig.l4.sport": 43562, "orig.l4.dport": 22, "orig.raw.pktlen": 0, "orig.raw.pktcount": 0, "reply.ip.protocol": 6, "reply.l4.sport": 22, "reply.l4.dport": 43562, "reply.raw.pktlen": 0, "reply.raw.pktcount": 0, "ct.mark": 0, "ct.id": 784049088, "ct.event": 1, "flow.start.sec": 1555408950, "flow.start.usec": 517694, "oob.family": 2, "oob.protocol": 0, "src_ip": "192.168.0.254", "dest_ip": "10.151.151.22", "reply.ip.saddr.str": "10.151.151.22", "reply.ip.daddr.str": "192.168.0.254"}
destruction event:
{"timestamp": "2019-04-16T12:05:00+0200", "dvc": "testdev", "orig.ip.protocol": 6, "orig.l4.sport": 43562, "orig.l4.dport": 22, "orig.raw.pktlen": 115789, "orig.raw.pktcount": 2011, "reply.ip.protocol": 6, "reply.l4.sport": 22, "reply.l4.dport": 43562, "reply.raw.pktlen": 28701079, "reply.raw.pktcount": 13119, "ct.mark": 0, "ct.id": 784049088, "ct.event": 4, "flow.start.sec": 4084, "flow.start.usec": 805056, "flow.end.sec": 1555409100, "flow.end.usec": 321544, "oob.family": 2, "oob.protocol": 0, "src_ip": "192.168.0.254", "dest_ip": "10.151.151.22", "reply.ip.saddr.str": "10.151.151.22", "reply.ip.daddr.str": "192.168.0.254"}
flow.start.sec is 4084 == Thu, 01 Jan 1970 02:08:04 +0100
Any ideas ?
Re: ulogd2 question - meaning of flow.start.sec when hash_mode == 0 (NFCT input, JSON output)
Quoted 32 lines · Michal Soltys
When using
hash_mode== 0 with defaultevent_mask, it looks like destruction entry has uninitialized (or having some specific meaning) flow.start.sec and flow.start.usec - as the former converts to early 1970.Peeking at the code in
event_handler_no_hashtable()it looks like it should be set correctly, but alas something seems amiss.creation event:
{"timestamp": "2019-04-16T12:02:30+0200", "dvc": "testdev", "orig.ip.protocol": 6, "orig.l4.sport": 43562, "orig.l4.dport": 22, "orig.raw.pktlen": 0, "orig.raw.pktcount": 0, "reply.ip.protocol": 6, "reply.l4.sport": 22, "reply.l4.dport": 43562, "reply.raw.pktlen": 0, "reply.raw.pktcount": 0, "ct.mark": 0, "ct.id": 784049088, "ct.event":
1, "flow.start.sec": 1555408950, "flow.start.usec": 517694, "oob.family": 2, "oob.protocol": 0, "src_ip": "192.168.0.254", "dest_ip": "10.151.151.22", "reply.ip.saddr.str": "10.151.151.22", "reply.ip.daddr.str": "192.168.0.254"}destruction event:
{"timestamp": "2019-04-16T12:05:00+0200", "dvc": "testdev", "orig.ip.protocol": 6, "orig.l4.sport": 43562, "orig.l4.dport": 22, "orig.raw.pktlen": 115789, "orig.raw.pktcount": 2011, "reply.ip.protocol": 6, "reply.l4.sport": 22, "reply.l4.dport": 43562, "reply.raw.pktlen": 28701079, "reply.raw.pktcount": 13119, "ct.mark": 0, "ct.id": 784049088, "ct.event": 4, "flow.start.sec": 4084, "flow.start.usec": 805056, "flow.end.sec": 1555409100, "flow.end.usec":
321544, "oob.family": 2, "oob.protocol": 0, "src_ip": "192.168.0.254", "dest_ip": "10.151.151.22", "reply.ip.saddr.str": "10.151.151.22", "reply.ip.daddr.str": "192.168.0.254"}
Quoted 4 lines
flow.start.sec is 4084 == Thu, 01 Jan 1970 02:08:04 +0100
Any ideas ?
For the record, the same unusual timestamp is shown when monitoring with conntrack:
[NEW] tcp 6 120 SYN_SENT src=192.168.0.254 dst=10.151.151.22 sport=44572 dport=22 [UNREPLIED] src=10.151.151.22 dst=192.168.0.254 sport=22 dport=44572 [UPDATE] tcp 6 60 SYN_RECV src=192.168.0.254 dst=10.151.151.22 sport=44572 dport=22 src=10.151.151.22 dst=192.168.0.254 sport=22 dport=44572 [UPDATE] tcp 6 432000 ESTABLISHED src=192.168.0.254 dst=10.151.151.22 sport=44572 dport=22 src=10.151.151.22 dst=192.168.0.254 sport=22 dport=44572 [ASSURED] [UPDATE] tcp 6 120 FIN_WAIT src=192.168.0.254 dst=10.151.151.22 sport=44572 dport=22 src=10.151.151.22 dst=192.168.0.254 sport=22 dport=44572 [ASSURED] [UPDATE] tcp 6 60 CLOSE_WAIT src=192.168.0.254 dst=10.151.151.22 sport=44572 dport=22 src=10.151.151.22 dst=192.168.0.254 sport=22 dport=44572 [ASSURED] [UPDATE] tcp 6 30 LAST_ACK src=192.168.0.254 dst=10.151.151.22 sport=44572 dport=22 src=10.151.151.22 dst=192.168.0.254 sport=22 dport=44572 [ASSURED] [UPDATE] tcp 6 120 TIME_WAIT src=192.168.0.254 dst=10.151.151.22 sport=44572 dport=22 src=10.151.151.22 dst=192.168.0.254 sport=22 dport=44572 [ASSURED] [DESTROY] tcp 6 src=192.168.0.254 dst=10.151.151.22 sport=44572 dport=22 packets=55 bytes=6577 src=10.151.151.22 dst=192.168.0.254 sport=22 dport=44572 packets=51 bytes=8919 [ASSURED] delta-time=1555405003 [start=Thu Jan 1 04:58:06 1970] [stop=Tue Apr 16 14:54:49 2019]
Florian Westphal
maintainerPATCH 1/1 netfilter: never get/set skb->tstamp
In reply to Michal Soltyssetting net.netfilter.nf_conntrack_timestamp=1 breaks xmit with fq scheduler. skb->tstamp might be "refreshed" using ktime_get_real(), but fq expects CLOCK_MONOTONIC.
This patch removes all places in netfilter that check/set skb->tstamp:
1. To fix the bogus "start" time seen with conntrack timestamping for outgoing packets, never use skb->tstamp and always use current time.
2. In nfqueue and nflog, only use skb->tstamp for incoming packets, as determined by current hook (prerouting, input, forward).
3. xt_time has to use system clock as well rather than skb->tstamp. We could still use skb->tstamp for prerouting/input/foward, but I see no advantage to make this conditional.
--- net/netfilter/nf_conntrack_core.c | 7 ++----- net/netfilter/nfnetlink_log.c | 2 +- net/netfilter/nfnetlink_queue.c | 2 +- net/netfilter/xt_time.c | 23 ++++++++++++++--------- 4 files changed, 18 insertions(+), 16 deletions(-)
net/netfilter/nf_conntrack_core.c +2 -5
@@ -1017,12 +1017,9 @@ __nf_conntrack_confirm(struct sk_buff *skb)/* set conntrack timestamp, if enabled. */ tstamp =nf_conn_tstamp_find (ct); -if (tstamp) { -if (skb->tstamp ==0 ) -__net_timestamp (skb); +if (tstamp) + tstamp->start =ktime_get_real_ns (); - tstamp->start =ktime_to_ns (skb->tstamp); - }/* Since the lookup is lockless, hash insertion must be done after * starting the timer and setting the CONFIRMED bit. The RCU barriers * guarantee that no other CPU can find the conntrack before the above
net/netfilter/nfnetlink_log.c +1 -1
@@ -540,7 +540,7 @@ __build_packet_message(struct nfnl_log_net *log, goto nla_put_failure; } -if (skb->tstamp) { +if (hooknum <=NF_INET_FORWARD && skb->tstamp) {struct nfulnl_msg_packet_timestamp ts;struct timespec64 kts =ktime_to_timespec64 (skb->tstamp); ts.sec =cpu_to_be64 (kts.tv_sec);
net/netfilter/nfnetlink_queue.c +1 -1
@@ -582,7 +582,7 @@ nfqnl_build_packet_message(struct net *net, struct nfqnl_instance *queue,if (nfqnl_put_bridge (entry, skb) <0 ) goto nla_put_failure; -if (entskb->tstamp) { +if (entry->state.hook <=NF_INET_FORWARD && entskb->tstamp) {struct nfqnl_msg_packet_timestamp ts;struct timespec64 kts =ktime_to_timespec64 (entskb->tstamp);
net/netfilter/xt_time.c +14 -9
@@ -163,19 +163,24 @@ time_mt(const struct sk_buff *skb, struct xt_action_param *par)s64 stamp;/* -* We cannot use get_seconds() instead of __net_timestamp() here. +* We need real time here, but we can neither use skb->tstamp +* nor __net_timestamp(). +* +* skb->tstamp and skb->skb_mstamp_ns overlap, however, they +* use different clock types (real vs monotonic). +* * Suppose you have two rules: -* 1. match before 13:00 -* 2. match after 13:00 +* 1. match before 13:00 +* 2. match after 13:00 +* * If you match against processing time (get_seconds) it * may happen that the same packet matches both rules if -* it arrived at the right moment before 13:00. +* it arrived at the right moment before 13:00, so it would be +* better to check skb->tstamp and set it via __net_timestamp() +* if needed. This however breaks outgoing packets tx timestamp, +* and causes them to get delayed forever by fq packet scheduler. */ -if (skb->tstamp ==0 ) -__net_timestamp ((struct sk_buff *)skb); - - stamp =ktime_to_ns (skb->tstamp); - stamp =div_s64 (stamp,NSEC_PER_SEC ); + stamp =get_seconds ();if (info->flags &XT_TIME_LOCAL_TZ )/* Adjust for local timezone */
Signature · 1 line
2.21.0
Re: [PATCH nf] netfilter: never get/set skb->tstamp
Quoted 15 lines · Florian Westphal
setting net.netfilter.nf_conntrack_timestamp=1 breaks xmit with fq scheduler.
skb->tstampmight be "refreshed" usingktime_get_real(), but fq expectsCLOCK_MONOTONIC.This patch removes all places in netfilter that check/set
skb->tstamp:1. To fix the bogus "start" time seen with conntrack timestamping for outgoing packets, never use
skb->tstampand always use current time.
2. In nfqueue and nflog, only useskb->tstampfor incoming packets, as determined by current hook (prerouting, input, forward).
3.xt_timehas to use system clock as well rather thanskb->tstamp. We could still useskb->tstampfor prerouting/input/foward, but I see no advantage to make this conditional.
Quoted 5 lines
Fixes fb420d5d91c1 ("tcp/fq: move back to CLOCK_MONOTONIC")Cc Eric Dumazet <[email protected]>Reported-by Michal Soltys <[email protected]>Signed-off-by Florian Westphal <[email protected]>---
SGTM, thanks for taking care of this Florian
Re: [PATCH nf] netfilter: never get/set skb->tstamp
In reply to Florian WestphalQuoted 14 lines · Florian Westphal
setting net.netfilter.nf_conntrack_timestamp=1 breaks xmit with fq scheduler.
skb->tstampmight be "refreshed" usingktime_get_real(), but fq expectsCLOCK_MONOTONIC.This patch removes all places in netfilter that check/set
skb->tstamp:1. To fix the bogus "start" time seen with conntrack timestamping for
outgoing packets, never use skb->tstamp and always use current time.2. In nfqueue and nflog, only use
skb->tstampfor incoming packets, as determined by current hook (prerouting, input, forward).
3.xt_timehas to use system clock as well rather thanskb->tstamp. We could still useskb->tstampfor prerouting/input/foward, but I see no advantage to make this conditional.
Quoted 4 lines
Fixes fb420d5d91c1 ("tcp/fq: move back to CLOCK_MONOTONIC")Cc Eric Dumazet <[email protected]>Reported-by Michal Soltys <[email protected]>Signed-off-by Florian Westphal <[email protected]>
Thanks for quick patch.
Applied it on top of 5.0.8 - wanted to confirm everything seems to be working fine (and timestamps are correct).
Re: [PATCH nf] netfilter: never get/set skb->tstamp
In reply to Florian WestphalQuoted 6 lines · Florian Westphal
setting net.netfilter.nf_conntrack_timestamp=1 breaks xmit with fq scheduler.
skb->tstampmight be "refreshed" usingktime_get_real(), but fq expectsCLOCK_MONOTONIC.This patch removes all places in netfilter that check/set
skb->tstamp:
Quoted 7 lines
1. To fix the bogus "start" time seen with conntrack timestamping for outgoing packets, never use
skb->tstampand always use current time.
2. In nfqueue and nflog, only useskb->tstampfor incoming packets, as determined by current hook (prerouting, input, forward).
3.xt_timehas to use system clock as well rather thanskb->tstamp. We could still useskb->tstampfor prerouting/input/foward, but I see no advantage to make this conditional.
Applied, thanks Florian.