GIT: [PATCH v3 0/3] netfilter trace infrastructure support
Subsystems (3)
Florian Westphal
maintainer3rd round.
See the individual patches for changelog vs. V 2.
Patrick: Nothing fancy from libnftnl side, just the 3 new attributes (oiftype, nfproto, policy).
On kernel side I addressed all of the comments.
In case of NFPROTO_INET, the new NFTA_TRACE_NFPROTO will be 'NFPROTO_IPV4' (or IPV6), the netlink header family is set to NFPROTO_INET.
I tried to reduce nft_do_chain bloat, +80 on my system. We have one unconditional foo = false assignment, all nftrace calls are avoided in 'no trace rules set' case via static key.
Let me know if you spot more problems.
Thanks,
Florian
Florian Westphal
maintainerPATCH v3 1/3 src: add trace infrastructure support
parses trace monitor netlink messages from the kernel and builds nftnl_trace struct that contains the dissected information.
Provides getters to access these attributes.
---
Changes since v2:
- add NFTNL_TRACE_NFPROTO, NFTNL_TRACE_POLICY, NFTNL_TRACE_OIFTYPE
- use VERDICT without changing it
- remove NFTNL_TRACE_QUEUE_ID
Changes since v1:
- removed all printf support, not needed by Patricks nft patch
- adjust for changed NFTA_TRACE* kernel attributes
- VERDICT is now nested, so treat it accordingly
- removed TRANSPORTPROTO, seems its not needed at this time
- add NFTNL_TRACE_QUEUE_ID to obtain nfqueue number
- NF_VERDICT is normalized in this case, i.e.
if verdict from kernel is '42 << 16 | NF_QUEUE', make NF_QUEUE the verdict and put the queue number ->queue_id, accessible via NFTNL_TRACE_QUEUE_ID.
include/libnftnl/Makefile.am | 1 + include/libnftnl/trace.h | 55 +++++ include/linux/netfilter/nf_tables.h | 52 +++++ src/Makefile.am | 1 + src/libnftnl.map | 15 ++ src/trace.c | 438 ++++++++++++++++++++++++++++++++++++ src/utils.c | 12 + 7 files changed, 574 insertions(+) create mode 100644 include/libnftnl/trace.h create mode 100644 src/trace.c
include/libnftnl/Makefile.am +1 -0
@@ -1,5 +1,6 @@ pkginclude_HEADERS = batch.h \ table.h \ + trace.h \ chain.h \ rule.h \ expr.h \
include/libnftnl/trace.h new file +55 -0
55 lines not on this page · raw message
include/linux/netfilter/nf_tables.h +52 -0
68 lines not on this page · raw message
src/Makefile.am +1 -0
@@ -11,6 +11,7 @@ libnftnl_la_SOURCES = utils.c \ common.c \ gen.c \ table.c \ + trace.c \ chain.c \ rule.c \ set.c \
src/libnftnl.map +15 -0
@@ -498,3 +498,18 @@ global: local: *; }; + +LIBNFTNL_4.1 { + nftnl_trace_alloc; + nftnl_trace_free; + + nftnl_trace_is_set; + + nftnl_trace_get_u16; + nftnl_trace_get_u32; + nftnl_trace_get_u64; + nftnl_trace_get_str; + nftnl_trace_get_data; + + nftnl_trace_nlmsg_parse; +} LIBNFTNL_4;
src/trace.c new file +438 -0
438 lines not on this page · raw message
src/utils.c +12 -0
@@ -146,12 +146,24 @@ const char *nftnl_verdict2str(uint32_t verdict)return "accept" ;case NF_DROP :return "drop" ; +case NF_STOLEN : +return "stolen" ; +case NF_QUEUE : +return "queue" ; +case NF_REPEAT : +return "repeat" ; +case NF_STOP : +return "stop" ;case NFT_RETURN :return "return" ;case NFT_JUMP :return "jump" ;case NFT_GOTO :return "goto" ; +case NFT_CONTINUE : +return "continue" ; +case NFT_BREAK : +return "break" ;default :return "unknown" ; }
Signature · 1 line
2.4.10
Pablo Neira Ayuso
maintainerRe: [PATCH v3 libnftnl 1/3] src: add trace infrastructure support
Quoted 4 lines · Florian Westphal
parses trace monitor netlink messages from the kernel and builds
nftnl_tracestruct that contains the dissected information.Provides getters to access these attributes.
I leave it to you to push trace code to libnftnl. Thanks.
Florian Westphal
maintainerRe: [PATCH v3 libnftnl 1/3] src: add trace infrastructure support
Quoted 7 lines · Pablo Neira Ayuso
Quoted 4 lines · Florian Westphal
parses trace monitor netlink messages from the kernel and builds
nftnl_tracestruct that contains the dissected information.Provides getters to access these attributes.
I leave it to you to push trace code to libnftnl. Thanks.
Pushed libnftnl changes, thanks Pablo.
@Patrick: I won't push anything trace related to nftables.git since I'm assuming that you'll take care of nftables side eventually.
Cheers,
Florian
Florian Westphal
maintainerPATCH v2 2/3 netfilter: nf_tables: extend tracing infrastructure
In reply to Florian Westphalnft monitor mode can then decode and display this trace data.
Parts of LL/Network/Transport headers are provided as separate attributes.
Otherwise, printing IP address data becomes virtually impossible for userspace since in the case of the netdev family we really don't want userspace to have to know all the possible link layer types and/or sizes just to display/print an ip address.
We also don't want userspace to have to follow ipv6 header chains to get the s/dport info, the kernel already did this work for us.
To avoid bloating nft_do_chain all data required for tracing is encapsulated in nft_traceinfo.
The structure is initialized unconditionally(!) for each nft_do_chain invocation.
This unconditionall call will be moved under a static key in a followup patch.
With lots of help from Patrick McHardy and Pablo Neira.
---
Changes since v2:
- set nfgen_family to the rule family, not pkt->pf. To get the 'logical' family (e.g. NFPROTO_INET), we keep basechain ptr in nft_traceinfo struct.
- export oif type, policy and pkt->pf
- miniminze bloating of nft_do_chain
- revert to a simpler id for now, based on skb address and rx hash
- several cosmetic changes
Changes since v1:
- move trace into new file, nf_tables_trace.c (Pablo)
- add new netlink group for trace messages (Pablo, Patrick)
- make verdict nested attr (Patrick)
- add percpu area to store current processed skb id (suggested by Patrick)
- fold arguments to nft_trace_notify into a structure, nft_traceinfo. This would allow easier extension later on, e.g. ptr to registers to e.g. log NFT_BREAK/mismatch as well.
include/net/netfilter/nf_tables.h | 32 ++++ include/uapi/linux/netfilter/nf_tables.h | 52 ++++++ include/uapi/linux/netfilter/nfnetlink.h | 2 + net/netfilter/Makefile | 2 +- net/netfilter/nf_tables_api.c | 12 +- net/netfilter/nf_tables_core.c | 45 ++++-- net/netfilter/nf_tables_trace.c | 269 +++++++++++++++++++++++++++++++ net/netfilter/nfnetlink.c | 1 + 8 files changed, 396 insertions(+), 19 deletions(-) create mode 100644 net/netfilter/nf_tables_trace.c
include/net/netfilter/nf_tables.h +32 -0
@@ -888,6 +888,38 @@ void nft_unregister_chain_type(const struct nf_chain_type *);int nft_register_expr (struct nft_expr_type *);void nft_unregister_expr (struct nft_expr_type *); +int nft_verdict_dump (struct sk_buff *skb,int type, +const struct nft_verdict *v); + +/** +* struct nft_traceinfo - nft tracing information and state +* +* @pkt: pktinfo currently processed +* @basechain: base chain currently processed +* @chain: chain currently processed +* @rule: rule that was evaluated +* @verdict: verdict given by rule +* @type: event type (enum nft_trace_types) +* @packet_dumped: packet headers sent in a previous traceinfo message +* @trace: other struct members are initialised +*/ +struct nft_traceinfo { +const struct nft_pktinfo *pkt; +const struct nft_base_chain *basechain; +const struct nft_chain *chain; +const struct nft_rule *rule; +const struct nft_verdict *verdict; +enum nft_trace_types type; +bool packet_dumped; +bool trace; +}; + +void nft_trace_init (struct nft_traceinfo *info,const struct nft_pktinfo *pkt, +const struct nft_verdict *verdict, +const struct nft_chain *basechain); + +void nft_trace_notify (struct nft_traceinfo *info); +#define nft_dereference (p) \ nfnl_dereference(p, NFNL_SUBSYS_NFTABLES)
include/uapi/linux/netfilter/nf_tables.h +52 -0
68 lines not on this page · raw message
include/uapi/linux/netfilter/nfnetlink.h +2 -0
@@ -22,6 +22,8 @@ enum nfnetlink_groups {#define NFNLGRP_NFTABLES NFNLGRP_NFTABLES NFNLGRP_ACCT_QUOTA,#define NFNLGRP_ACCT_QUOTA NFNLGRP_ACCT_QUOTA + NFNLGRP_NFTRACE, +#define NFNLGRP_NFTRACE NFNLGRP_NFTRACE __NFNLGRP_MAX, };#define NFNLGRP_MAX (__NFNLGRP_MAX - 1)
net/netfilter/Makefile +1 -1
@@ -67,7 +67,7 @@ obj-$(CONFIG_NF_NAT_TFTP) += nf_nat_tftp.o obj-$(CONFIG_NETFILTER_SYNPROXY) += nf_synproxy_core.o # nf_tables -nf_tables-objs += nf_tables_core.o nf_tables_api.o +nf_tables-objs += nf_tables_core.o nf_tables_api.o nf_tables_trace.o nf_tables-objs += nft_immediate.o nft_cmp.o nft_lookup.o nft_dynset.o nf_tables-objs += nft_bitwise.o nft_byteorder.o nft_payload.o
net/netfilter/nf_tables_api.c +6 -6
@@ -4446,22 +4446,22 @@ static void nft_verdict_uninit(const struct nft_data *data) } } -static int nft_verdict_dump (struct sk_buff *skb,const struct nft_data *data) +int nft_verdict_dump (struct sk_buff *skb,int type,const struct nft_verdict *v) {struct nlattr *nest; - nest =nla_nest_start (skb,NFTA_DATA_VERDICT ); + nest =nla_nest_start (skb, type);if (!nest) goto nla_put_failure; -if (nla_put_be32 (skb,NFTA_VERDICT_CODE ,htonl (data->verdict.code))) +if (nla_put_be32 (skb,NFTA_VERDICT_CODE ,htonl (v->code))) goto nla_put_failure; -switch (data->verdict.code) { +switch (v->code) {case NFT_JUMP :case NFT_GOTO :if (nla_put_string (skb,NFTA_VERDICT_CHAIN , - data->verdict.chain->name)) + v->chain->name)) goto nla_put_failure; }nla_nest_end (skb, nest);
@@ -4572,7 +4572,7 @@ int nft_data_dump(struct sk_buff *skb, int attr, const struct nft_data *data, err =nft_value_dump (skb, data, len);break ;case NFT_DATA_VERDICT : - err =nft_verdict_dump (skb, data); + err =nft_verdict_dump (skb,NFTA_DATA_VERDICT , &data->verdict);break ;default : err = -EINVAL ;
net/netfilter/nf_tables_core.c +33 -12
97 lines not on this page · raw message
net/netfilter/nf_tables_trace.c new file +269 -0
269 lines not on this page · raw message
net/netfilter/nfnetlink.c +1 -0
@@ -49,6 +49,7 @@ static const int nfnl_group2type[NFNLGRP_MAX+1] = { [NFNLGRP_CONNTRACK_EXP_DESTROY] = NFNL_SUBSYS_CTNETLINK_EXP, [NFNLGRP_NFTABLES] = NFNL_SUBSYS_NFTABLES, [NFNLGRP_ACCT_QUOTA] = NFNL_SUBSYS_ACCT, + [NFNLGRP_NFTRACE] = NFNL_SUBSYS_NFTABLES, };void nfnl_lock (__u8 subsys_id)
Signature · 1 line
2.4.10
Re: [PATCH v2 nf-next 2/3] netfilter: nf_tables: extend tracing infrastructure
Quoted 19 lines · Florian Westphal
nft monitor mode can then decode and display this trace data.
Parts of LL/Network/Transport headers are provided as separate attributes.
Otherwise, printing IP address data becomes virtually impossible for userspace since in the case of the netdev family we really don't want userspace to have to know all the possible link layer types and/or sizes just to display/print an ip address.
We also don't want userspace to have to follow ipv6 header chains to get the s/dport info, the kernel already did this work for us.
To avoid bloating
nft_do_chainall data required for tracing is encapsulated innft_traceinfo.The structure is initialized unconditionally(!) for each
nft_do_chaininvocation.
Quoted 6 lines
This unconditionall call will be moved under a static key in a followup patch.
With lots of help from Patrick McHardy and Pablo Neira.
Signed-off-by Florian Westphal <[email protected]>
Looks very good to me, nice work!
for both kernel patches.
Florian Westphal
maintainerPATCH v3 3/3 netfilter: nf_tables: wrap tracing with a static key
In reply to Florian WestphalOnly needed when meta nftrace rule(s) were added.
The assumption is that no such rules are active, so the call to nft_trace_init is "never" needed.
When nftrace rules are active, we always call the nft_trace_* functions, but will only send netlink messages when all of the following are true:
- traceinfo structure was initialised
- skb->nf_trace == 1
- at least one subscriber to trace group.
Adding an extra conditional
(static_branch ... && skb->nf_trace)
nft_trace_init( ..)
Is possible but results in a larger nft_do_chain footprint.
---
Changes since v2:
- avoid bloat by always calling __nft_trace_packet once static key is active.
Changes since v1:
- do not wrap rulenum++ into static key, because it might get incremented too late when trace rules are added/removed at high rate, giving wrong results.
- add EXPORT_SYMBOL for nft_meta_set_destroy.
include/net/netfilter/nf_tables_core.h | 1 + include/net/netfilter/nft_meta.h | 3 +++ net/bridge/netfilter/nft_meta_bridge.c | 1 + net/netfilter/nf_tables_core.c | 9 ++++++--- net/netfilter/nf_tables_trace.c | 4 ++++ net/netfilter/nft_meta.c | 16 ++++++++++++++++ 6 files changed, 31 insertions(+), 3 deletions(-)
include/net/netfilter/nf_tables_core.h +1 -0
@@ -57,6 +57,7 @@ struct nft_payload_set { };extern const struct nft_expr_ops nft_payload_fast_ops; +extern struct static_key_false nft_trace_enabled;int nft_payload_module_init (void );void nft_payload_module_exit (void );
include/net/netfilter/nft_meta.h +3 -0
@@ -33,4 +33,7 @@ void nft_meta_set_eval(const struct nft_expr *expr,struct nft_regs *regs, const struct nft_pktinfo *pkt); +void nft_meta_set_destroy (const struct nft_ctx *ctx, +const struct nft_expr *expr); +#endif
net/bridge/netfilter/nft_meta_bridge.c +1 -0
@@ -84,6 +84,7 @@ static const struct nft_expr_ops nft_meta_bridge_set_ops = { .size = NFT_EXPR_SIZE(sizeof(struct nft_meta)), .eval = nft_meta_set_eval, .init = nft_meta_set_init, + .destroy = nft_meta_set_destroy, .dump = nft_meta_set_dump, };
net/netfilter/nf_tables_core.c +6 -3
@@ -16,6 +16,7 @@#include <linux/skbuff.h> #include <linux/netlink.h> #include <linux/netfilter.h> +#include <linux/static_key.h> #include <linux/netfilter/nfnetlink.h> #include <linux/netfilter/nf_tables.h> #include <net/netfilter/nf_tables_core.h>
@@ -50,7 +51,7 @@ static noinline void __nft_trace_packet(struct nft_traceinfo *info, {const struct nft_pktinfo *pkt = info->pkt; -if (!pkt->skb->nf_trace) +if (!info->trace || !pkt->skb->nf_trace)return ; info->chain = chain;
@@ -70,7 +71,7 @@ static inline void nft_trace_packet(struct nft_traceinfo *info,int rulenum, enum nft_trace_types type) { - if (unlikely(info->trace)) { + if (static_branch_unlikely(&nft_trace_enabled)) { info->rule = rule;__nft_trace_packet (info, chain, rulenum, type); }
@@ -137,7 +138,9 @@ nft_do_chain(struct nft_pktinfo *pkt, void *priv)unsigned int gencursor =nft_genmask_cur (net);struct nft_traceinfo info; -nft_trace_init (&info, pkt, ®s.verdict, basechain); + info.trace = false; +if (static_branch_unlikely (&nft_trace_enabled)) +nft_trace_init (&info, pkt, ®s.verdict, basechain); do_chain: rulenum =0 ; rule = list_entry(&chain->rules, struct nft_rule, list);
net/netfilter/nf_tables_trace.c +4 -0
@@ -8,6 +8,7 @@*/ #include <linux/module.h> +#include <linux/static_key.h> #include <linux/hash.h> #include <linux/jhash.h> #include <linux/if_vlan.h>
@@ -24,6 +25,9 @@#define NFT_TRACETYPE_NETWORK_HSIZE 40#define NFT_TRACETYPE_TRANSPORT_HSIZE 20 +DEFINE_STATIC_KEY_FALSE (nft_trace_enabled); +EXPORT_SYMBOL_GPL (nft_trace_enabled); +static int trace_fill_id (struct sk_buff *nlskb,struct sk_buff *skb) {__be32 id;
net/netfilter/nft_meta.c +16 -0
@@ -18,10 +18,12 @@#include <linux/ip.h> #include <linux/ipv6.h> #include <linux/smp.h> +#include <linux/static_key.h> #include <net/dst.h> #include <net/sock.h> #include <net/tcp_states.h> /* for TCP_TIME_WAIT */ #include <net/netfilter/nf_tables.h> +#include <net/netfilter/nf_tables_core.h> #include <net/netfilter/nft_meta.h> void nft_meta_get_eval(const struct nft_expr *expr,
@@ -297,6 +299,9 @@ int nft_meta_set_init(const struct nft_ctx *ctx,if (err <0 )return err; +if (priv->key ==NFT_META_NFTRACE ) +static_branch_inc (&nft_trace_enabled); +return 0 ; }EXPORT_SYMBOL_GPL (nft_meta_set_init);
@@ -334,6 +339,16 @@ nla_put_failure: }EXPORT_SYMBOL_GPL (nft_meta_set_dump); +void nft_meta_set_destroy (const struct nft_ctx *ctx, +const struct nft_expr *expr) +{ +const struct nft_meta *priv =nft_expr_priv (expr); + +if (priv->key ==NFT_META_NFTRACE ) +static_branch_dec (&nft_trace_enabled); +} +EXPORT_SYMBOL_GPL (nft_meta_set_destroy); +static struct nft_expr_type nft_meta_type;static const struct nft_expr_ops nft_meta_get_ops = { .type = &nft_meta_type,
@@ -348,6 +363,7 @@ static const struct nft_expr_ops nft_meta_set_ops = { .size = NFT_EXPR_SIZE(sizeof(struct nft_meta)), .eval = nft_meta_set_eval, .init = nft_meta_set_init, + .destroy = nft_meta_set_destroy, .dump = nft_meta_set_dump, };
Signature · 1 line
2.4.10
Quoted 11 lines · Florian Westphal
3rd round.
See the individual patches for changelog vs. V 2.
Patrick: Nothing fancy from libnftnl side, just the 3 new attributes (oiftype, nfproto, policy).
On kernel side I addressed all of the comments.
In case of
NFPROTO_INET, the newNFTA_TRACE_NFPROTOwill be 'NFPROTO_IPV4' (or IPV6), the netlink header family is set toNFPROTO_INET.
That's sounds good.
Quoted 1 line
I tried to reduce
nft_do_chainbloat, +80 on my system.
That's even better than I expected, thanks!
Quoted 4 lines
We have one unconditional foo = false assignment, all nftrace calls are avoided in 'no trace rules set' case via static key.
Let me know if you spot more problems.
I'll review it tommorrow if you don't mind, currently trying to wrap my head around the sub-byte and offset expressions. We have some limitations which I'd like to address if I'm already working in that area before I continue on the tracing part.