Skip to contentSkip to conversation

src: add conntrack information to trace monitor mode

PATCH Superseded Florian Westphal 2 participants 7 messages Jul 7, 2025

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Florian Westphal

author maintainer
#0 lore raw

cover src: add conntrack information to trace monitor mode

First patch is a preparation patch that moves the trace code from netlink.c to the new trace.c file.

Second patch adds the ct info to the trace output.

This patch exposes the 'clash' bit to userspace. (Technically its the kernel counterpart).

If you dislike this, I can send a kernel patch that removes the bit before dumping ct status bits to userspace, let me know.

Change since v1:
- prep patch to split to trace.c
- add ct status fields to data types

Florian Westphal (2):
src: split monitor trace code into new trace.c
src: add conntrack information to trace monitor mode

 Makefile.am                                   |   1 +
 doc/data-types.txt                            |  31 +-
 include/linux/netfilter/nf_conntrack_common.h |  16 +
 include/netlink.h                             |   5 -
 include/trace.h                               |   8 +
 src/ct.c                                      |   8 +
 src/monitor.c                                 |   2 +-
 src/netlink.c                                 | 332 -------------
 src/trace.c                                   | 462 ++++++++++++++++++
 9 files changed, 512 insertions(+), 353 deletions(-)
 create mode 100644 include/trace.h
 create mode 100644 src/trace.c
Signature · 1 line
2.49.0
author maintainer
#1 lore raw

PATCH 1/2 src: split monitor trace code into new trace.c

Preparation patch to avoid putting more trace functionality into netlink.c.

---
 Makefile.am       |   1 +
 include/netlink.h |   5 -
 include/trace.h   |   8 ++
 src/monitor.c     |   2 +-
 src/netlink.c     | 332 -------------------------------------------
 src/trace.c       | 353 ++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 363 insertions(+), 338 deletions(-)
 create mode 100644 include/trace.h
 create mode 100644 src/trace.c
Makefile.am +1 -0
index fb64105dda88..ba09e7f0953d 100644
@@ -220,6 +220,7 @@ src_libnftables_la_SOURCES = \
 	src/misspell.c \
 	src/mnl.c \
 	src/monitor.c \
+	src/trace.c \
 	src/netlink.c \
 	src/netlink_delinearize.c \
 	src/netlink_linearize.c \
include/netlink.h +0 -5
index c7da6f9e3bcb..2737d5708b29 100644
@@ -227,11 +227,6 @@ struct ruleset_parse {
 	struct cmd              *cmd;
 };
 
-struct nftnl_parse_ctx;
-
-int netlink_events_trace_cb(const struct nlmsghdr *nlh, int type,
-			    struct netlink_mon_handler *monh);
-
 enum nft_data_types dtype_map_to_kernel(const struct datatype *dtype);
 
 void netlink_linearize_init(struct netlink_linearize_ctx *lctx,
include/trace.h new file +8 -0
new file mode 100644
index 000000000000..ebebb47d5c66
@@ -0,0 +1,8 @@
+#ifndef NFTABLES_TRACE_H
+#define NFTABLES_TRACE_H
+#include <linux/netlink.h>
+
+struct netlink_mon_handler;
+int netlink_events_trace_cb(const struct nlmsghdr *nlh, int type,
+			    struct netlink_mon_handler *monh);
+#endif /* NFTABLES_TRACE_H */
src/monitor.c +1 -1
index e3e38c2a12b7..83977907e266 100644
@@ -16,7 +16,6 @@
 #include <inttypes.h>
 
 #include <libnftnl/table.h>
-#include <libnftnl/trace.h>
 #include <libnftnl/chain.h>
 #include <libnftnl/expr.h>
 #include <libnftnl/object.h>
@@ -32,6 +31,7 @@
 #include <nftables.h>
 #include <netlink.h>
 #include <mnl.h>
+#include <trace.h>
 #include <expression.h>
 #include <statement.h>
 #include <gmputil.h>
src/netlink.c +0 -332

347 lines not on this page · raw message

src/trace.c new file +353 -0

353 lines not on this page · raw message

Signature · 1 line
2.49.0
author maintainer
#3 lore raw

PATCH 2/2 src: add conntrack information to trace monitor mode

In reply to Florian Westphal

Upcoming kernel change provides the packets conntrack state in the trace message data.

This allows to see if packet is seen as original or reply, the conntrack state (new, establieshed, related) and the status bits which show if e.g. NAT was applied. Alsoi include conntrack ID so users can use conntrack tool to query the kernel for more information via ctnetlink.

This improves debugging when e.g. packets do not pick up the expected NAT mapping, which could e.g. also happen because of expectations following the NAT binding of the owning conntrack entry.

Example output ("conntrack: " lines are new):

trace id 32 t PRE_RAW packet: iif "enp0s3" ether saddr [..]
trace id 32 t PRE_RAW rule tcp flags syn meta nftrace set 1 (verdict continue)
trace id 32 t PRE_RAW policy accept
trace id 32 t PRE_MANGLE conntrack: ct direction original ct state new ct id 2641368242 trace id 32 t PRE_MANGLE packet: iif "enp0s3" ether saddr [..]
trace id 32 t ct_new_pre rule jump rpfilter (verdict jump rpfilter)
trace id 32 t PRE_MANGLE policy accept
trace id 32 t INPUT conntrack: ct direction original ct state new ct status dnat-done ct id 2641368242 trace id 32 t INPUT packet: iif "enp0s3" [..]
trace id 32 t public_in rule tcp dport 443 accept (verdict accept)

v2: add more status bits: helper, clash, offload, hw-offload.
add flag explanation to documentation.

---
 doc/data-types.txt                            |  31 ++---
 include/linux/netfilter/nf_conntrack_common.h |  16 +++
 src/ct.c                                      |   8 ++
 src/trace.c                                   | 109 ++++++++++++++++++
 4 files changed, 149 insertions(+), 15 deletions(-)
doc/data-types.txt +16 -15
index 46b0867cb5a4..d688350c4afd 100644
@@ -378,21 +378,22 @@ For each of the types above, keywords are available for convenience:
 .conntrack status (ct_status)
 [options="header"]
 |==================
-|Keyword| Value
-|expected|
-1
-|seen-reply|
-2
-|assured|
-4
-|confirmed|
-8
-|snat|
-16
-|dnat|
-32
-|dying|
-512
+|Keyword| Value	| Description
+|expected|1| Expected connection; conntrack helper set it up
+|seen-reply|2| Conntrack has seen packets in both directions
+|assured| 4 |Conntrack entry will not be removed if hash table is full
+|confirmed | 8 | Initial packet processed
+|snat| 16 | Original source address differs from reply destination
+|dnat| 32 | Original destination differs from reply source
+|seq-adjust| 64 | tcp sequence number rewrite due to conntrack helper or synproxy
+|snat-done| 128 | tried to find matching snat/masquerade rule
+|dnat-done| 256 | tried to find matching dnat/redirect rule
+|dying| 512 | Connection about to be deleted
+|fixed-timeout | 1024 | entry expires even if traffic is active
+|clash  | 4096 | packet drop avoidance scheme
+|helper | 8192 | connection is monitored by conntrack helper
+|offload | 16384 | connection is offloaded to a flow table
+|hw-offload | 32768 | connection is offloaded to hardware
 |================
 
 .conntrack event bits (ct_event)
include/linux/netfilter/nf_conntrack_common.h +16 -0
index 768ff251308b..22bbb6c92fb1 100644
@@ -77,6 +77,22 @@ enum ip_conntrack_status {
 	/* Connection has fixed timeout. */
 	IPS_FIXED_TIMEOUT_BIT = 10,
 	IPS_FIXED_TIMEOUT = (1 << IPS_FIXED_TIMEOUT_BIT),
+
+	/* Conntrack is a fake untracked entry.  Obsolete and not used anymore */
+	IPS_UNTRACKED_BIT = 12,
+	IPS_UNTRACKED = (1 << IPS_UNTRACKED_BIT),
+
+	/* Conntrack got a helper explicitly attached (ruleset, ctnetlink). */
+	IPS_HELPER_BIT = 13,
+	IPS_HELPER = (1 << IPS_HELPER_BIT),
+
+	/* Conntrack has been offloaded to flow table. */
+	IPS_OFFLOAD_BIT = 14,
+	IPS_OFFLOAD = (1 << IPS_OFFLOAD_BIT),
+
+	/* Conntrack has been offloaded to hardware. */
+	IPS_HW_OFFLOAD_BIT = 15,
+	IPS_HW_OFFLOAD = (1 << IPS_HW_OFFLOAD_BIT),
 };
 
 /* Connection tracking event types */
src/ct.c +8 -0
index 71ebb9483893..7fa808800fcf 100644
@@ -98,7 +98,15 @@ static const struct symbol_table ct_status_tbl = {
 		SYMBOL("confirmed",	IPS_CONFIRMED),
 		SYMBOL("snat",		IPS_SRC_NAT),
 		SYMBOL("dnat",		IPS_DST_NAT),
+		SYMBOL("seq-adjust",	IPS_SEQ_ADJUST),
+		SYMBOL("snat-done",	IPS_SRC_NAT_DONE),
+		SYMBOL("dnat-done",	IPS_DST_NAT_DONE),
 		SYMBOL("dying",		IPS_DYING),
+		SYMBOL("fixed-timeout",	IPS_FIXED_TIMEOUT),
+		SYMBOL("clash",		IPS_UNTRACKED_BIT),
+		SYMBOL("helper",	IPS_HELPER_BIT),
+		SYMBOL("offload",	IPS_OFFLOAD_BIT),
+		SYMBOL("hw-offload",	IPS_HW_OFFLOAD_BIT),
 		SYMBOL_LIST_END
 	},
 };
src/trace.c +109 -0

121 lines not on this page · raw message

Signature · 1 line
2.49.0
#4 lore raw
In reply to Florian Westphal
Quoted 11 lines · Florian Westphal

First patch is a preparation patch that moves the trace code from netlink.c to the new trace.c file.

Second patch adds the ct info to the trace output.

This patch exposes the 'clash' bit to userspace. (Technically its the kernel counterpart).

If you dislike this, I can send a kernel patch that removes the bit before dumping ct status bits to userspace, let me know.

If this is intentional, then

+ SYMBOL("clash", IPS_UNTRACKED_BIT),

hiding clash bit is probably a good idea.

Just hide it from userspace nftables in this series, later I'd suggest you proceed with the kernel update.

Thanks.

author maintainer
#5 lore raw
Quoted 13 lines · Pablo Neira Ayuso
Quoted 11 lines · Florian Westphal

First patch is a preparation patch that moves the trace code from netlink.c to the new trace.c file.

Second patch adds the ct info to the trace output.

This patch exposes the 'clash' bit to userspace. (Technically its the kernel counterpart).

If you dislike this, I can send a kernel patch that removes the bit before dumping ct status bits to userspace, let me know.

Quoted 5 lines

If this is intentional, then

+ SYMBOL("clash", IPS_UNTRACKED_BIT),

hiding clash bit is probably a good idea.

Currently the existence of 'clash' entries are a kernel-internal implemenation detail.

Neither /proc or ctnetlink exposes them, the dump handlers only emit ORIGINAL direction, but the clash entries are only inserted into the hashes for the reply tuple.

Hence, they are not visible so far.
With this change however, a packet that matches a clash entry (reply dir), will have skb->_nfct set to a 'clash' entry and so its ct->status and ID are exposed to userspace.

This isn't a problem, but it does mean that the IPS_UNTRACKED_BIT is set in ct->status.

IPS_UNTRACKED isn't used anymore in the kernel, it has been re-purposed to flag the clash entries (IPS_NAT_CLASH_BIT = IPS_UNTRACKED_BIT, but the former constant isn't exposed via UAPI).

Thats the reason for this awkward

SYMBOL("clash", IPS_UNTRACKED_BIT),

Quoted 2 lines

Just hide it from userspace nftables in this series, later I'd suggest you proceed with the kernel update.

If I remove this line from the patch, then I can skip/ignore the value in userspace, e.g.:

src/trace.c +1 -1
index b270951025b8..b3b2c8fdf1b9 100644
@@ -264,7 +264,7 @@ static struct expr *trace_alloc_list(const struct datatype *dtype,
        for (i = 0; i < 32; i++) {
                uint32_t bitv = v & (1 << i);
 
-               if (bitv == 0)
+               if (bitv == 0 || i == IPS_UNTRACKED_BIT)
                        continue;
 

and remove the IPS_UNTRACKED_BIT from the symbol table.

Then followup with a kernel patch that removes IPS_UNTRACKED_BIT before dumping ct->status.

Does that sound ok?
If so, I'll apply the first patch in this series before resending 2/2.

Thanks!

#6 lore raw
Quoted 14 lines · Florian Westphal
src/trace.c +1 -1
index b270951025b8..b3b2c8fdf1b9 100644
@@ -264,7 +264,7 @@ static struct expr *trace_alloc_list(const struct datatype *dtype,
        for (i = 0; i < 32; i++) {
                uint32_t bitv = v & (1 << i);
 
-               if (bitv == 0)
+               if (bitv == 0 || i == IPS_UNTRACKED_BIT)
                        continue;
 

and remove the IPS_UNTRACKED_BIT from the symbol table.

Quoted 4 lines

Then followup with a kernel patch that removes IPS_UNTRACKED_BIT before dumping ct->status.

Does that sound ok?

Yes, let's keep it back until there is a clear use-case for this, either not exposing it or promoting it as a first class citizen.

Quoted 1 line

If so, I'll apply the first patch in this series before resending 2/2.

Thanks.