src: trace infrastructure support
Subsystems (3)
Florian Westphal
maintainerv2 cover src: trace infrastructure support
This is V2 of the nft trace patch set.
See the individual patches for a change vs. v1 description.
Patrick: I did a quick test w. your last public patches (plus a few minor changes to make it build again) and things seem to work fine.
What does not work correctly is decoding of vlan header. I think whats happening is that we cannot splice sub-byte sized quantities from a larger chunk.
Other than that, things appear to work correctly.
Cheers,
Florian
Florian Westphal
maintainerPATCH v2 1/3 src: add trace infrastructure support
parses trace monitor netlink messages from the kernel and builds nftnl_trace struct that contains the dissected information.
Provides getters to access these attributes.
---
Changes since v1:
- removed all printf support, not needed by Patricks nft patch
- adjust for changed NFTA_TRACE* kernel attributes
- VERDICT is now nested, so treat it accordingly
- removed TRANSPORTPROTO, seems its not needed at this time
- add NFTNL_TRACE_QUEUE_ID to obtain nfqueue number
- NF_VERDICT is normalized in this case, i.e.
if verdict from kernel is '42 << 16 | NF_QUEUE', make NF_QUEUE the verdict and put the queue number ->queue_id, accessible via NFTNL_TRACE_QUEUE_ID.
include/libnftnl/Makefile.am | 1 + include/libnftnl/trace.h | 52 +++++ include/linux/netfilter/nf_tables.h | 46 +++++ src/Makefile.am | 1 + src/libnftnl.map | 15 ++ src/trace.c | 400 ++++++++++++++++++++++++++++++++++++ src/utils.c | 12 ++ 7 files changed, 527 insertions(+) create mode 100644 include/libnftnl/trace.h create mode 100644 src/trace.c
include/libnftnl/Makefile.am +1 -0
@@ -1,5 +1,6 @@ pkginclude_HEADERS = batch.h \ table.h \ + trace.h \ chain.h \ rule.h \ expr.h \
include/libnftnl/trace.h new file +52 -0
52 lines not on this page · raw message
include/linux/netfilter/nf_tables.h +46 -0
62 lines not on this page · raw message
src/Makefile.am +1 -0
@@ -11,6 +11,7 @@ libnftnl_la_SOURCES = utils.c \ common.c \ gen.c \ table.c \ + trace.c \ chain.c \ rule.c \ set.c \
src/libnftnl.map +15 -0
@@ -498,3 +498,18 @@ global: local: *; }; + +LIBNFTNL_4.1 { + nftnl_trace_alloc; + nftnl_trace_free; + + nftnl_trace_is_set; + + nftnl_trace_get_u16; + nftnl_trace_get_u32; + nftnl_trace_get_u64; + nftnl_trace_get_str; + nftnl_trace_get_data; + + nftnl_trace_nlmsg_parse; +} LIBNFTNL_4;
src/trace.c new file +400 -0
400 lines not on this page · raw message
src/utils.c +12 -0
@@ -146,12 +146,24 @@ const char *nftnl_verdict2str(uint32_t verdict)return "accept" ;case NF_DROP :return "drop" ; +case NF_STOLEN : +return "stolen" ; +case NF_QUEUE : +return "queue" ; +case NF_REPEAT : +return "repeat" ; +case NF_STOP : +return "stop" ;case NFT_RETURN :return "return" ;case NFT_JUMP :return "jump" ;case NFT_GOTO :return "goto" ; +case NFT_CONTINUE : +return "continue" ; +case NFT_BREAK : +return "break" ;default :return "unknown" ; }
Signature · 1 line
2.4.10
Re: [PATCH v2 libnftnl 1/3] src: add trace infrastructure support
Quoted 60 lines · Florian Westphal
- add
NFTNL_TRACE_QUEUE_IDto obtain nfqueue number
-NF_VERDICTis normalized in this case, i.e.
if verdict from kernel is '42 << 16 |NF_QUEUE', makeNF_QUEUEthe verdict and put the queue number ->queue_id, accessible viaNFTNL_TRACE_QUEUE_ID.src/trace.c new file +54
-0+static int nftnl_trace_parse_verdict_cb (const struct nlattr *attr,void *data) +{ +int type =mnl_attr_get_type (attr); +const struct nlattr **tb = data; + +switch (type) { +case NFTA_VERDICT_CODE : +if (mnl_attr_validate (attr,MNL_TYPE_U32 ) <0 ) +abi_breakage (); + tb[type] = attr; +break ; +case NFTA_VERDICT_CHAIN : +if (mnl_attr_validate (attr,MNL_TYPE_STRING ) <0 ) +abi_breakage (); + tb[type] = attr; +break ; + } + +return MNL_CB_OK ; +} + +static void +nftnl_trace_parse_verdict (const struct nlattr *attr,struct nftnl_trace *t) +{ +struct nlattr *tb[NFTA_VERDICT_MAX +1 ]; + +mnl_attr_parse_nested (attr, nftnl_trace_parse_verdict_cb, tb); + +if (!tb[NFTA_VERDICT_CODE ]) +abi_breakage (); + + t->verdict =ntohl (mnl_attr_get_u32 (tb[NFTA_VERDICT_CODE ])); + t->flags |= (1 <<NFTNL_TRACE_VERDICT ); + +switch (t->verdict) { +case NFT_GOTO :/* fallthough */ +case NFT_JUMP : +if (!tb[NFTA_VERDICT_CHAIN ]) +abi_breakage (); + t->jump_target =strdup (mnl_attr_get_str (tb[NFTA_VERDICT_CHAIN ])); +if (t->jump_target) + t->flags |= (1 <<NFTNL_TRACE_JUMP_TARGET ); +break ; +case NFT_RETURN :/* all other NFT_* cases fall through */ +case NFT_CONTINUE : +case NFT_BREAK : +break ; +case NF_ACCEPT :/* standard verdicts fall though */ +case NF_DROP : +case NF_STOLEN : +case NF_REPEAT : +case NF_STOP : +break ; +default :/* Unknown NF_ verdict, or verdict contains extra data */
src/trace.c line 326
Quoted 10 lines
src/trace.c new file +10
-0+switch (t->verdict &NF_VERDICT_MASK ) { +case NF_QUEUE : + t->queue_id = t->verdict >>16 ; + t->verdict =NF_QUEUE ; + t->flags |= (1 <<NFTNL_TRACE_QUEUE_ID ); +break ; + } +break ; + } +}
This doesn't handle NF_DROP_ERRNO() codes. I would suggest using nftnl_parse_verdict() and struct nftnl_data_reg so we have a common interface for verdicts. Mid term I'd prefer if we pass around verdict structs and have an interface for those instead of having IMM_VERDICT/IMM_CHAIN, SET_ELEM_VERDICT/SET_ELEM_CHAIN, ...
Regarding the NFTNL_TRACE_QUEUE_ID, I'm not convinced this is a good way to go. We'd need another NFTNL_TRACE_DROP_VERDICT_ERRNO or something like that, and the user needs to make another distinction on which further attributes to get based on the verdict itself. It seems easier to just decode it in the user and don't try to put too much knowledge in the library, it's after all just meant for communication.
Florian Westphal
maintainerPATCH v2 2/3 netfilter: nf_tables: extend tracing infrastructure
In reply to Florian Westphalnft monitor mode can then decode and display this trace data.
Parts of LL/Network/Transport headers are provided as separate attributes.
Otherwise, printing IP address data becomes virtually impossible for userspace since in the case of the netdev family we really don't want userspace to have to know all the possible link layer types and/or sizes just to display/print an ip address.
We also don't want userspace to have to follow ipv6 header chains to get the s/dport info, the kernel already did this work for us.
---
Changes since v1:
- move trace into new file, nf_tables_trace.c (Pablo)
- add new netlink group for trace messages (Pablo, Patrick)
- make verdict nested attr (Patrick)
- add percpu area to store current processed skb id (suggested by Patrick)
- fold arguments to nft_trace_notify into a structure, nft_traceinfo. This would allow easier extension later on, e.g. ptr to registers to e.g. log NFT_BREAK/mismatch as well.
include/net/netfilter/nf_tables.h | 17 +++ include/uapi/linux/netfilter/nf_tables.h | 46 ++++++ include/uapi/linux/netfilter/nfnetlink.h | 2 + net/netfilter/Makefile | 2 +- net/netfilter/nf_tables_api.c | 12 +- net/netfilter/nf_tables_core.c | 35 ++++- net/netfilter/nf_tables_trace.c | 253 +++++++++++++++++++++++++++++++ net/netfilter/nfnetlink.c | 1 + net/netfilter/nft_meta.c | 4 +- 9 files changed, 356 insertions(+), 16 deletions(-) create mode 100644 net/netfilter/nf_tables_trace.c
include/net/netfilter/nf_tables.h +17 -0
@@ -888,6 +888,23 @@ void nft_unregister_chain_type(const struct nf_chain_type *);int nft_register_expr (struct nft_expr_type *);void nft_unregister_expr (struct nft_expr_type *); +int nft_verdict_dump (struct sk_buff *skb, +const struct nft_verdict *v,u16 type); + +struct nft_traceinfo { +const struct nft_chain *chain; +const struct nft_rule *rule; +const struct nft_verdict *verdict; +enum nft_trace_types type; + +bool packet_dumped; +}; + +void nft_trace_notify (struct nft_traceinfo *info, +const struct nft_pktinfo *pkt); + +void nft_trace_start (struct sk_buff *skb); +#define nft_dereference (p) \ nfnl_dereference(p, NFNL_SUBSYS_NFTABLES)
include/uapi/linux/netfilter/nf_tables.h +46 -0
62 lines not on this page · raw message
include/uapi/linux/netfilter/nfnetlink.h +2 -0
@@ -22,6 +22,8 @@ enum nfnetlink_groups {#define NFNLGRP_NFTABLES NFNLGRP_NFTABLES NFNLGRP_ACCT_QUOTA,#define NFNLGRP_ACCT_QUOTA NFNLGRP_ACCT_QUOTA + NFNLGRP_NFTRACE, +#define NFNLGRP_NFTRACE NFNLGRP_NFTRACE __NFNLGRP_MAX, };#define NFNLGRP_MAX (__NFNLGRP_MAX - 1)
net/netfilter/Makefile +1 -1
@@ -67,7 +67,7 @@ obj-$(CONFIG_NF_NAT_TFTP) += nf_nat_tftp.o obj-$(CONFIG_NETFILTER_SYNPROXY) += nf_synproxy_core.o # nf_tables -nf_tables-objs += nf_tables_core.o nf_tables_api.o +nf_tables-objs += nf_tables_core.o nf_tables_api.o nf_tables_trace.o nf_tables-objs += nft_immediate.o nft_cmp.o nft_lookup.o nft_dynset.o nf_tables-objs += nft_bitwise.o nft_byteorder.o nft_payload.o
net/netfilter/nf_tables_api.c +6 -6
@@ -4446,22 +4446,22 @@ static void nft_verdict_uninit(const struct nft_data *data) } } -static int nft_verdict_dump (struct sk_buff *skb,const struct nft_data *data) +int nft_verdict_dump (struct sk_buff *skb,const struct nft_verdict *v,u16 type) {struct nlattr *nest; - nest =nla_nest_start (skb,NFTA_DATA_VERDICT ); + nest =nla_nest_start (skb, type);if (!nest) goto nla_put_failure; -if (nla_put_be32 (skb,NFTA_VERDICT_CODE ,htonl (data->verdict.code))) +if (nla_put_be32 (skb,NFTA_VERDICT_CODE ,htonl (v->code))) goto nla_put_failure; -switch (data->verdict.code) { +switch (v->code) {case NFT_JUMP :case NFT_GOTO :if (nla_put_string (skb,NFTA_VERDICT_CHAIN , - data->verdict.chain->name)) + v->chain->name)) goto nla_put_failure; }nla_nest_end (skb, nest);
@@ -4572,7 +4572,7 @@ int nft_data_dump(struct sk_buff *skb, int attr, const struct nft_data *data, err =nft_value_dump (skb, data, len);break ;case NFT_DATA_VERDICT : - err =nft_verdict_dump (skb, data); + err =nft_verdict_dump (skb, &data->verdict,NFTA_DATA_VERDICT );break ;default : err = -EINVAL ;
net/netfilter/nf_tables_core.c +27 -8
@@ -54,12 +54,23 @@ static void __nft_trace_packet(const struct nft_pktinfo *pkt, rulenum); } -static inline void nft_trace_packet (const struct nft_pktinfo *pkt, +static inline void nft_trace_packet (struct nft_traceinfo *info, +const struct nft_pktinfo *pkt,const struct nft_chain *chain, -int rulenum,enum nft_trace type) +const struct nft_rule *rule, +const struct nft_verdict *verdict, +int rulenum, +enum nft_trace_types type) { -if (unlikely (pkt->skb->nf_trace)) +if (unlikely (pkt->skb->nf_trace)) { + info->chain = chain; + info->rule = rule; + info->verdict = verdict; + info->type = type; +nft_trace_notify (info, pkt); +__nft_trace_packet (pkt, chain, rulenum, type); + } } static void nft_cmp_fast_eval(const struct nft_expr *expr,
@@ -121,7 +132,9 @@ nft_do_chain(struct nft_pktinfo *pkt, void *priv)struct nft_stats *stats;int rulenum;unsigned int gencursor =nft_genmask_cur (net); +struct nft_traceinfo info; + info.packet_dumped = false; do_chain: rulenum =0 ; rule = list_entry(&chain->rules, struct nft_rule, list);
@@ -151,7 +164,8 @@ next_rule: regs.verdict.code =NFT_CONTINUE ;continue ;case NFT_CONTINUE : -nft_trace_packet (pkt, chain, rulenum,NFT_TRACE_RULE ); +nft_trace_packet (&info, pkt, chain, rule, ®s.verdict, + rulenum,NFT_TRACETYPE_RULE );continue ; }break ;
@@ -161,7 +175,8 @@ next_rule:case NF_ACCEPT :case NF_DROP :case NF_QUEUE : -nft_trace_packet (pkt, chain, rulenum,NFT_TRACE_RULE ); +nft_trace_packet (&info, pkt, chain, rule, ®s.verdict, + rulenum,NFT_TRACETYPE_RULE );return regs.verdict.code; }
@@ -174,7 +189,8 @@ next_rule: stackptr++;/* fall through */ case NFT_GOTO : -nft_trace_packet (pkt, chain, rulenum,NFT_TRACE_RULE ); +nft_trace_packet (&info, pkt, chain, rule, ®s.verdict, + rulenum,NFT_TRACETYPE_RULE ); chain = regs.verdict.chain; goto do_chain;
@@ -182,7 +198,9 @@ next_rule: rulenum++;/* fall through */ case NFT_RETURN : -nft_trace_packet (pkt, chain, rulenum,NFT_TRACE_RETURN ); +if (stackptr) +nft_trace_packet (&info, pkt, chain, rule, ®s.verdict, + rulenum,NFT_TRACETYPE_RETURN );break ;default :WARN_ON (1 );
@@ -196,7 +214,8 @@ next_rule: goto next_rule; } -nft_trace_packet (pkt, basechain,-1 ,NFT_TRACE_POLICY ); +nft_trace_packet (&info, pkt, basechain,NULL ,NULL ,-1 , +NFT_TRACETYPE_POLICY );rcu_read_lock_bh (); stats =this_cpu_ptr (rcu_dereference (nft_base_chain (basechain)->stats));
net/netfilter/nf_tables_trace.c new file +253 -0
253 lines not on this page · raw message
net/netfilter/nfnetlink.c +1 -0
@@ -49,6 +49,7 @@ static const int nfnl_group2type[NFNLGRP_MAX+1] = { [NFNLGRP_CONNTRACK_EXP_DESTROY] = NFNL_SUBSYS_CTNETLINK_EXP, [NFNLGRP_NFTABLES] = NFNL_SUBSYS_NFTABLES, [NFNLGRP_ACCT_QUOTA] = NFNL_SUBSYS_ACCT, + [NFNLGRP_NFTRACE] = NFNL_SUBSYS_NFTABLES, };void nfnl_lock (__u8 subsys_id)
net/netfilter/nft_meta.c +3 -1
@@ -204,7 +204,9 @@ void nft_meta_set_eval(const struct nft_expr *expr, skb->priority = value;break ;case NFT_META_NFTRACE : - skb->nf_trace =1 ; +if (skb->nf_trace) +break ; +nft_trace_start (pkt->skb);break ;default :WARN_ON (1 );
Signature · 1 line
2.4.10
Re: [PATCH v2 nf-next 2/3] netfilter: nf_tables: extend tracing infrastructure
Quoted 1 line · Florian Westphal
nft monitor mode can then decode and display this trace data.
First round of comments below:
include/net/netfilter/nf_tables.h void nft_unregister_chain_type(const struct nf_chain_type *); · line 892
Quoted 7 lines
include/net/netfilter/nf_tables.h +2
-0+int nft_verdict_dump (struct sk_buff *skb, +const struct nft_verdict *v,u16 type);
We usually use "struct sk_buff *skb, int attr, ...". I know int is actually not the correct type but that's what all the nla_ functions use. Let's keep it consistent please.
include/net/netfilter/nf_tables.h void nft_unregister_chain_type(const struct nf_chain_type *); · line 901
Quoted 8 lines
include/net/netfilter/nf_tables.h +8
-0+struct nft_traceinfo { +const struct nft_chain *chain; +const struct nft_rule *rule; +const struct nft_verdict *verdict; +enum nft_trace_types type; + +bool packet_dumped; +};
Also consistent style please, all structs in that file indent the field names neatly and they are usually also commented.
include/net/netfilter/nf_tables.h next_rule: · line 218
Quoted 8 lines
+++
b/net/netfilter/nf_tables_core.cinclude/net/netfilter/nf_tables.h +2
-1@@ -196,7 +214,8 @@ next_rule: goto next_rule; } -nft_trace_packet (pkt, basechain,-1 ,NFT_TRACE_POLICY ); +nft_trace_packet (&info, pkt, basechain,NULL ,NULL ,-1 , +NFT_TRACETYPE_POLICY );
We unfortunately don't get the actual policy this way. I know we don't have a verdict struct available, but would be nice to fix this.
Quoted 8 lines
include/net/netfilter/nf_tables.h +8
-0+++ b/net/netfilter/nf_tables_trace.c +static int nf_trace_fill_meta_info (struct sk_buff *nlskb, +const struct nft_pktinfo *pkt) +{ +if (pkt->in) { +if (nla_put_be32 (nlskb,NFTA_TRACE_IIF , +htonl (pkt->in->ifindex))) +return -1 ;
include/net/netfilter/nf_tables.h next_rule:
Quoted 10 lines
include/net/netfilter/nf_tables.h +10
-0+ +/* needed for LL_HEADER decode */ +if (nla_put_be16 (nlskb,NFTA_TRACE_IIFTYPE , +htons (pkt->in->type))) +return -1 ; + } + +if (pkt->out && +nla_put_be32 (nlskb,NFTA_TRACE_OIF ,htonl (pkt->out->ifindex))) +return -1 ;
I think we might also need the OIFTYPE and the hook number for full coverage. On the output path we don't have an iif but still might have an LL header.
include/net/netfilter/nf_tables.h
Quoted 7 lines
include/net/netfilter/nf_tables.h +7
-0+static int nf_trace_fill_chain_info (struct sk_buff *nlskb, +const struct nft_chain *chain) +{ +if (nla_put_string (nlskb,NFTA_TRACE_TABLE , chain->table->name)) +return -1 ; + +return nla_put_string (nlskb,NFTA_TRACE_CHAIN , chain->name);
I think we also need the netfilter family here. We get pkt->pf, but that might differ from the table/chain family in case of NFPROTO_INET. Right now we get IPv4/v6 and are unable to look up any objects since we're searching in the wrong family.
include/net/netfilter/nf_tables.h
Quoted 5 lines
include/net/netfilter/nf_tables.h +5
-0+void nft_trace_start (struct sk_buff *skb) +{ +u32 seq =atomic_inc_return (&trace_cnt) <<NFT_ID_SKB_HASHBITS ; + +__this_cpu_write (trace_seq, seq);
So we're using a global counter to initialize the per CPU vars? That doesn't seem to make sense. I was thinking about partitioning the ID space among CPUs in advance, this global counter kind of defeats using per CPU at all.
Ok I think I see what you're doing here, you want a stable ID while the packet is processed. The problem is we can not really guarantee that for the output path with involuntary preemption, between different hook invocations another packet might come along.
I actually didn't consider that we need to get the same ID multiple times when I suggested the per CPU counter, I was only thinking about getting a non-clashing ID *once*. I think we need to come up with a different idea, sorry about that.
Quoted 3 lines
include/net/netfilter/nf_tables.h +3
-0+ skb->nf_trace =1 ; +} +EXPORT_SYMBOL (nft_trace_start);
Florian Westphal
maintainerRe: [PATCH v2 nf-next 2/3] netfilter: nf_tables: extend tracing infrastructure
Quoted 7 lines · Patrick McHardy
include/net/netfilter/nf_tables.h void nft_unregister_chain_type(const struct nf_chain_type *); · line 892
Quoted 2 lines · Florian Westphal
include/net/netfilter/nf_tables.h +2
-0+int nft_verdict_dump (struct sk_buff *skb, +const struct nft_verdict *v,u16 type);We usually use "struct
sk_buff*skb, int attr, ...". I know int is actually not the correct type but that's what all thenla_functions use. Let's keep it consistent please.
OK.
Quoted 11 lines
include/net/netfilter/nf_tables.h void nft_unregister_chain_type(const struct nf_chain_type *); · line 901
Quoted 8 lines
include/net/netfilter/nf_tables.h +8
-0+struct nft_traceinfo { +const struct nft_chain *chain; +const struct nft_rule *rule; +const struct nft_verdict *verdict; +enum nft_trace_types type; + +bool packet_dumped; +};Also consistent style please, all structs in that file indent the field names neatly and they are usually also commented.
Fair enough.
Quoted 11 lines
net/netfilter/nf_tables_core.c next_rule: · line 218
Quoted 8 lines
+++
b/net/netfilter/nf_tables_core.cnet/netfilter/nf_tables_core.c +2
-1@@ -196,7 +214,8 @@ next_rule: goto next_rule; } -nft_trace_packet (pkt, basechain,-1 ,NFT_TRACE_POLICY ); +nft_trace_packet (&info, pkt, basechain,NULL ,NULL ,-1 , +NFT_TRACETYPE_POLICY );We unfortunately don't get the actual policy this way. I know we don't have a verdict struct available, but would be nice to fix this.
Suggestions? Should I add a new attribute or pretend policy was a verdict? (i.e., add fake verdict struct and set verdict code to policy)?
Quoted 6 lines
net/netfilter/nf_tables_trace.c line 88
Quoted 3 lines
net/netfilter/nf_tables_trace.c new file +3
-0+if (pkt->out && +nla_put_be32 (nlskb,NFTA_TRACE_OIF ,htonl (pkt->out->ifindex))) +return -1 ;I think we might also need the OIFTYPE and the hook number for full coverage. On the output path we don't have an iif but still might have an LL header.
Okay. Any preference on how to do this?
I.e, should I include OIFTYPE only if no indev is available?
Alternatives would be to include both and let userspace decide or use more generic DEV_TYPE attr that is set to indev->type, and, if no indev is there, outdev->type?
Quoted 10 lines
net/netfilter/nf_tables_trace.c line 159
Quoted 7 lines
net/netfilter/nf_tables_trace.c new file +7
-0+static int nf_trace_fill_chain_info (struct sk_buff *nlskb, +const struct nft_chain *chain) +{ +if (nla_put_string (nlskb,NFTA_TRACE_TABLE , chain->table->name)) +return -1 ; + +return nla_put_string (nlskb,NFTA_TRACE_CHAIN , chain->name);I think we also need the netfilter family here. We get
pkt->pf, but that might differ from the table/chain family in case ofNFPROTO_INET.
Good point, will add NFTA_TRACE_FAMILY
Quoted 13 lines
now we get IPv4/v6 and are unable to look up any objects since we're searching in the wrong family.
net/netfilter/nf_tables_trace.c line 250
Quoted 5 lines
net/netfilter/nf_tables_trace.c new file +5
-0+void nft_trace_start (struct sk_buff *skb) +{ +u32 seq =atomic_inc_return (&trace_cnt) <<NFT_ID_SKB_HASHBITS ; + +__this_cpu_write (trace_seq, seq);So we're using a global counter to initialize the per CPU vars? That doesn't seem to make sense. I was thinking about partitioning the ID space among CPUs in advance, this global counter kind of defeats using per CPU at all.
Quoted 2 lines
Ok I think I see what you're doing here, you want a stable ID while the packet is processed.
Yes, that was the idea -- which is why v1 used hash_ptr(skb)
Quoted 3 lines
The problem is we can not really guarantee that for
the output path with involuntary preemption, between different hook invocations another packet might come along.
Yes.
Quoted 3 lines
I actually didn't consider that we need to get the same ID multiple times when I suggested the per CPU counter, I was only thinking about getting a non-clashing ID *once*.
Oh. What would be the purpose of that?
The idea was to allow to easily figure out which trace messages belong to the same packet. (i.e., don't make it difficult to dissect events from different packets being processed on other cpus...)
Quoted 2 lines
I think we need to come up with a different idea,
sorry about that.
Hmm, in that case I have no idea how to fix this, sorry. Only *stable* id is hashing skb address, but as you pointed out that causes fast id reuse
(I think that would be ok, since its good enough to tell
near-simultaneous trace events apart).
Re: [PATCH v2 nf-next 2/3] netfilter: nf_tables: extend tracing infrastructure
Quoted 12 lines · Florian Westphal
Quoted 10 lines · Patrick McHardy
net/netfilter/nf_tables_core.c next_rule: · line 218
Quoted 7 lines
net/netfilter/nf_tables_core.c +2
-1@@ -196,7 +214,8 @@ next_rule: goto next_rule; } -nft_trace_packet (pkt, basechain,-1 ,NFT_TRACE_POLICY ); +nft_trace_packet (&info, pkt, basechain,NULL ,NULL ,-1 , +NFT_TRACETYPE_POLICY );We unfortunately don't get the actual policy this way. I know we don't have a verdict struct available, but would be nice to fix this.
Quoted 2 lines
Suggestions? Should I add a new attribute or pretend policy was a verdict? (i.e., add fake verdict struct and set verdict code to policy)?
I don't see any other alternative right now.
Quoted 7 lines
Quoted 6 lines
net/netfilter/nf_tables_trace.c line 88
Quoted 3 lines
net/netfilter/nf_tables_trace.c new file +3
-0+if (pkt->out && +nla_put_be32 (nlskb,NFTA_TRACE_OIF ,htonl (pkt->out->ifindex))) +return -1 ;I think we might also need the OIFTYPE and the hook number for full coverage. On the output path we don't have an iif but still might have an LL header.
Quoted 6 lines
Okay. Any preference on how to do this?
I.e, should I include OIFTYPE only if no indev is available?Alternatives would be to include both and let userspace decide or use more generic
DEV_TYPEattr that is set toindev->type, and, if no indev is there,outdev->type?
One of both *seems* fine. If we have an iif, the header is still based on the received packet. If we only have an oif, the header, if present, will be based on that type, so I think that should work.
Quoted 11 lines
Quoted 10 lines
net/netfilter/nf_tables_trace.c line 159
Quoted 7 lines
net/netfilter/nf_tables_trace.c new file +7
-0+static int nf_trace_fill_chain_info (struct sk_buff *nlskb, +const struct nft_chain *chain) +{ +if (nla_put_string (nlskb,NFTA_TRACE_TABLE , chain->table->name)) +return -1 ; + +return nla_put_string (nlskb,NFTA_TRACE_CHAIN , chain->name);I think we also need the netfilter family here. We get
pkt->pf, but that might differ from the table/chain family in case ofNFPROTO_INET.
Quoted 1 line
Good point, will add
NFTA_TRACE_FAMILY
Actually thinking about that, all the other nftables messages use the netfilter family for nfgen_family, so for consistency we should do that as well and put pkt->pf in the new attribute.
Quoted 24 lines
Quoted 13 lines
now we get IPv4/v6 and are unable to look up any objects since we're searching in the wrong family.
net/netfilter/nf_tables_trace.c line 250
Quoted 5 lines
net/netfilter/nf_tables_trace.c new file +5
-0+void nft_trace_start (struct sk_buff *skb) +{ +u32 seq =atomic_inc_return (&trace_cnt) <<NFT_ID_SKB_HASHBITS ; + +__this_cpu_write (trace_seq, seq);So we're using a global counter to initialize the per CPU vars? That doesn't seem to make sense. I was thinking about partitioning the ID space among CPUs in advance, this global counter kind of defeats using per CPU at all.
Quoted 2 lines
Ok I think I see what you're doing here, you want a stable ID while the packet is processed.
Yes, that was the idea -- which is why v1 used
hash_ptr(skb)Quoted 3 lines
The problem is we can not really guarantee that for
the output path with involuntary preemption, between different hook invocations another packet might come along.Yes.
Quoted 8 lines
Quoted 3 lines
I actually didn't consider that we need to get the same ID multiple times when I suggested the per CPU counter, I was only thinking about getting a non-clashing ID *once*.
Oh. What would be the purpose of that?
The idea was to allow to easily figure out which trace messages belong to the same packet. (i.e., don't make it difficult to dissect events from different packets being processed on other cpus...)
Yeah, I didn't consider that we need to get the same identifier in multiple hook invocations.
Quoted 9 lines
Quoted 2 lines
I think we need to come up with a different idea,
sorry about that.Hmm, in that case I have no idea how to fix this, sorry. Only *stable* id is hashing skb address, but as you pointed out that causes fast id reuse
(I think that would be ok, since its good enough to tell
near-simultaneous trace events apart).
I also don't have a better suggestion right now. We could try to find additional skb members that are stable within netfilter, but other than that I have no better ideas.
Re: [PATCH v2 nf-next 2/3] netfilter: nf_tables: extend tracing infrastructure
In reply to Florian WestphalSome more comments on use of the netlink functions below:
Quoted 7 lines · Florian Westphal
+++
b/net/netfilter/nf_tables_trace.cnet/netfilter/nf_tables_trace.c new file +5
-0@@ -0,0 +1,253 @@ +static int trace_fill_header (struct sk_buff *nlskb,u16 type, +const struct sk_buff *skb, +int off,unsigned int len) +{ +struct nlattr *nla;
net/netfilter/nf_tables_trace.c line 15
Quoted 10 lines
net/netfilter/nf_tables_trace.c new file +10
-0+ +if (len ==0 ) +return 0 ; + +if (skb_tailroom (nlskb) <nla_total_size (len)) +return -1 ; + + nla = (struct nlattr *)skb_put (nlskb,nla_total_size (len)); + nla->nla_type = type; + nla->nla_len =nla_attr_size (len);
nla_reserve()? That will also take care of zeroing the padding.
Quoted 25 lines
net/netfilter/nf_tables_trace.c new file +25
-0+ +if (skb_copy_bits (skb, off,nla_data (nla), len)) +return -1 ; + +return 0 ; +} + +static int nf_trace_fill_meta_info (struct sk_buff *nlskb, +const struct nft_pktinfo *pkt) +{ +if (pkt->in) { +if (nla_put_be32 (nlskb,NFTA_TRACE_IIF , +htonl (pkt->in->ifindex))) +return -1 ; + +/* needed for LL_HEADER decode */ +if (nla_put_be16 (nlskb,NFTA_TRACE_IIFTYPE , +htons (pkt->in->type))) +return -1 ; + } + +if (pkt->out && +nla_put_be32 (nlskb,NFTA_TRACE_OIF ,htonl (pkt->out->ifindex))) +return -1 ; +
net/netfilter/nf_tables_trace.c line 99
Quoted 10 lines
net/netfilter/nf_tables_trace.c new file +10
-0+return 0 ; +} + +static int nf_trace_fill_ll_header (struct sk_buff *nlskb, +const struct sk_buff *skb) +{ +struct vlan_ethhdr veth; +unsigned int len, off; + + off =skb_mac_header (skb) - skb->data;
off is negative or zero, it should use a signed int I think.
net/netfilter/nf_tables_trace.c line 109
Quoted 10 lines
net/netfilter/nf_tables_trace.c new file +10
-0+ +if (skb_copy_bits (skb, off, &veth,ETH_HLEN )) +return -1 ; + + veth.h_vlan_proto = skb->vlan_proto; + veth.h_vlan_TCI =htons (skb_vlan_tag_get (skb)); + veth.h_vlan_encapsulated_proto = skb->protocol; + + len = min_t(unsigned int, + skb->data -skb_mac_header (skb),NFT_TRACETYPE_LL_HSIZE );
min_t(unsigned int, -off, NFT_TRACETYPE_LL_HSIZE) ?
net/netfilter/nf_tables_trace.c line 112
Quoted 3 lines
net/netfilter/nf_tables_trace.c new file +3
-0+ +if (WARN_ON_ONCE (len !=ETH_HLEN )) +return -1 ;
Why the min_t at all if we fail on != ETH_HLEN? I'd simply check that it is exactly ETH_HLEN before even starting copying.
net/netfilter/nf_tables_trace.c line 118
Quoted 6 lines
net/netfilter/nf_tables_trace.c new file +6
-0+ + len =sizeof (veth); +if (skb_tailroom (nlskb) <nla_total_size (len)) +return -1 ; + +return nla_put (nlskb,NFTA_TRACE_LL_HEADER , len, &veth);
nla_put() already checks the tailroom.
Quoted 20 lines
net/netfilter/nf_tables_trace.c new file +20
-0+} + +static int nf_trace_fill_pkt_info (struct sk_buff *nlskb, +const struct nft_pktinfo *pkt) +{ +const struct sk_buff *skb = pkt->skb; + unsigned int len = min_t(unsigned int, + pkt->xt.thoff -skb_network_offset (skb), +NFT_TRACETYPE_NETWORK_HSIZE ); +int off =skb_network_offset (skb); + +if (trace_fill_header (nlskb,NFTA_TRACE_NETWORK_HEADER , skb, off, len)) +return -1 ; + + len = min_t(unsigned int, skb->len - pkt->xt.thoff, +NFT_TRACETYPE_TRANSPORT_HSIZE ); + +if (trace_fill_header (nlskb,NFTA_TRACE_TRANSPORT_HEADER , skb, + pkt->xt.thoff, len)) +return -1 ;
net/netfilter/nf_tables_trace.c line 148
Quoted 10 lines
net/netfilter/nf_tables_trace.c new file +10
-0+ +if (!skb_mac_header_was_set (skb)) +return 0 ; + +if (skb_vlan_tag_get (skb)) +return nf_trace_fill_ll_header (nlskb, skb); + + len = min_t(unsigned int, + skb->data -skb_mac_header (skb),NFT_TRACETYPE_LL_HSIZE ); + off =skb_mac_header (skb) - skb->data;
Could again avoid the double calculation by using -off in min_t().
net/netfilter/nf_tables_trace.c line 159
Quoted 11 lines
net/netfilter/nf_tables_trace.c new file +11
-0+return trace_fill_header (nlskb,NFTA_TRACE_LL_HEADER , + skb, off, len); +} + +static int nf_trace_fill_chain_info (struct sk_buff *nlskb, +const struct nft_chain *chain) +{ +if (nla_put_string (nlskb,NFTA_TRACE_TABLE , chain->table->name)) +return -1 ; + +return nla_put_string (nlskb,NFTA_TRACE_CHAIN , chain->name);
I would not move simple stuff that is only used once like this to another function. Easier to read if you don't have to jump around in the code.
Quoted 93 lines
net/netfilter/nf_tables_trace.c new file +93
-0+} + +void nft_trace_notify (struct nft_traceinfo *info, +const struct nft_pktinfo *pkt) +{ +struct nfgenmsg *nfmsg; +struct nlmsghdr *nlh; +struct sk_buff *skb; +unsigned int size; +int event = (NFNL_SUBSYS_NFTABLES <<8 ) |NFT_MSG_TRACE ; + +if (!nfnetlink_has_listeners (pkt->net,NFNLGRP_NFTRACE )) +return ; + + size =nlmsg_total_size (sizeof (struct nfgenmsg )) + +nla_total_size (NFT_TABLE_MAXNAMELEN ) + +nla_total_size (NFT_CHAIN_MAXNAMELEN ) + +nla_total_size (sizeof (__be64)) +/* rule handle */ +nla_total_size (sizeof (__be32)) +/* trace type */ +nla_total_size (0 ) +/* VERDICT, nested */ +nla_total_size (sizeof (u32)) +/* verdict code */ +nla_total_size (NFT_CHAIN_MAXNAMELEN ) +/* jump target */ +nla_total_size (sizeof (u32)) +/* id */ +nla_total_size (NFT_TRACETYPE_LL_HSIZE ) + +nla_total_size (NFT_TRACETYPE_NETWORK_HSIZE ) + +nla_total_size (NFT_TRACETYPE_TRANSPORT_HSIZE ) + +nla_total_size (sizeof (u32)) +/* mark */ +nla_total_size (sizeof (u32)) +/* iif */ +nla_total_size (sizeof (u32)) +/* oif */ +nla_total_size (sizeof (__be16));/* device type */ + + skb =nlmsg_new (size,GFP_ATOMIC ); +if (!skb) +return ; + + nlh =nlmsg_put (skb,0 ,0 , event,sizeof (struct nfgenmsg ),0 ); +if (!nlh) + goto nla_put_failure; + + nfmsg =nlmsg_data (nlh); + nfmsg->nfgen_family = pkt->pf; + nfmsg->version =NFNETLINK_V0 ; + nfmsg->res_id =0 ; + +if (nla_put_be32 (skb,NFTA_TRACE_TYPE ,htonl (info->type))) + goto nla_put_failure; + +if (trace_fill_id (skb, (unsigned long )pkt->skb)) + goto nla_put_failure; + +if (info->chain && +nf_trace_fill_chain_info (skb, info->chain)) + goto nla_put_failure; + +if (info->rule && +nla_put_be64 (skb,NFTA_TRACE_RULE_HANDLE , +cpu_to_be64 (info->rule->handle))) + goto nla_put_failure; + +if ((info->type ==NFT_TRACETYPE_RETURN || + info->type ==NFT_TRACETYPE_RULE ) && +nft_verdict_dump (skb, info->verdict,NFTA_TRACE_VERDICT )) + goto nla_put_failure; + +if (pkt->skb->mark && +nla_put_be32 (skb,NFTA_TRACE_MARK ,htonl (pkt->skb->mark))) + goto nla_put_failure; + +if (!info->packet_dumped) { +if (nf_trace_fill_meta_info (skb, pkt)) + goto nla_put_failure; + +if (nf_trace_fill_pkt_info (skb, pkt)) + goto nla_put_failure; + info->packet_dumped = true; + } + +nlmsg_end (skb, nlh); +nfnetlink_send (skb, pkt->net,0 ,NFNLGRP_NFTRACE ,0 ,GFP_ATOMIC ); +return ; + + nla_put_failure: +WARN_ON_ONCE (1 ); +kfree_skb (skb); +} + +void nft_trace_start (struct sk_buff *skb) +{ +u32 seq =atomic_inc_return (&trace_cnt) <<NFT_ID_SKB_HASHBITS ; + +__this_cpu_write (trace_seq, seq); + skb->nf_trace =1 ; +}
Florian Westphal
maintainerRe: [PATCH v2 nf-next 2/3] netfilter: nf_tables: extend tracing infrastructure
Quoted 12 lines · Patrick McHardy
Quoted 11 lines · Florian Westphal
+++
b/net/netfilter/nf_tables_trace.cnet/netfilter/nf_tables_trace.c new file +9
-0@@ -0,0 +1,253 @@ +static int trace_fill_header (struct sk_buff *nlskb,u16 type, +const struct sk_buff *skb, +int off,unsigned int len) +{ +struct nlattr *nla; + +if (len ==0 ) +return 0 ; +
Quoted 8 lines
net/netfilter/nf_tables_trace.c line 64
Quoted 6 lines
net/netfilter/nf_tables_trace.c new file +6
-0+if (skb_tailroom (nlskb) <nla_total_size (len)) +return -1 ; + + nla = (struct nlattr *)skb_put (nlskb,nla_total_size (len)); + nla->nla_type = type; + nla->nla_len =nla_attr_size (len);
nla_reserve()? That will also take care of zeroing the padding.
Fixed, thanks.
Quoted 9 lines
net/netfilter/nf_tables_trace.c line 99
Quoted 7 lines
net/netfilter/nf_tables_trace.c new file +7
-0+static int nf_trace_fill_ll_header (struct sk_buff *nlskb, +const struct sk_buff *skb) +{ +struct vlan_ethhdr veth; +unsigned int len, off; + + off =skb_mac_header (skb) - skb->data;off is negative or zero, it should use a signed int I think.
Right, fixed.
Quoted 12 lines
net/netfilter/nf_tables_trace.c line 109
Quoted 10 lines
net/netfilter/nf_tables_trace.c new file +10
-0+ +if (skb_copy_bits (skb, off, &veth,ETH_HLEN )) +return -1 ; + + veth.h_vlan_proto = skb->vlan_proto; + veth.h_vlan_TCI =htons (skb_vlan_tag_get (skb)); + veth.h_vlan_encapsulated_proto = skb->protocol; + + len = min_t(unsigned int, + skb->data -skb_mac_header (skb),NFT_TRACETYPE_LL_HSIZE );
min_t(unsigned int, -off, NFT_TRACETYPE_LL_HSIZE)?
[..]
Quoted 2 lines
Why the
min_tat all if we fail on !=ETH_HLEN? I'd simply check that it is exactlyETH_HLENbefore even starting copying.
Good point, I removed min test completely and changed it to:
int off;
BUILD_BUG_ON(NFT_TRACETYPE_LL_HSIZE < ETH_HLEN);
off = skb_mac_header(skb) - skb->data;
if (len != -ETH_HLEN)
return -1;
...Quoted 8 lines
net/netfilter/nf_tables_trace.c line 118
Quoted 6 lines
net/netfilter/nf_tables_trace.c new file +6
-0+ + len =sizeof (veth); +if (skb_tailroom (nlskb) <nla_total_size (len)) +return -1 ; + +return nla_put (nlskb,NFTA_TRACE_LL_HEADER , len, &veth);
nla_put()already checks the tailroom.
Right, my bad. Removed.
Quoted 5 lines
net/netfilter/nf_tables_trace.c line 148
Quoted 3 lines
net/netfilter/nf_tables_trace.c new file +3
-0+ len = min_t(unsigned int, + skb->data -skb_mac_header (skb),NFT_TRACETYPE_LL_HSIZE ); + off =skb_mac_header (skb) - skb->data;Could again avoid the double calculation by using -off in
min_t().
Fixed.
Quoted 10 lines
net/netfilter/nf_tables_trace.c line 159
Quoted 7 lines
net/netfilter/nf_tables_trace.c new file +7
-0+static int nf_trace_fill_chain_info (struct sk_buff *nlskb, +const struct nft_chain *chain) +{ +if (nla_put_string (nlskb,NFTA_TRACE_TABLE , chain->table->name)) +return -1 ; + +return nla_put_string (nlskb,NFTA_TRACE_CHAIN , chain->name);I would not move simple stuff that is only used once like this to another function. Easier to read if you don't have to jump around in the code.
Ok, open-coded this instead.
Thanks for the review, I'll send a v3 soon.
Florian Westphal
maintainerPATCH v2 3/3 netfilter: nf_tables: wrap tracing with a static key
In reply to Florian WestphalOnly needed when meta nftrace rule(s) were added.
---
Changes since v1:
- do not wrap rulenum++ into static key, because it might get incremented too late when trace rules are added/removed at high rate, giving wrong results.
- add EXPORT_SYMBOL for nft_meta_set_destroy.
There was another followup patch to disable old tracing infrastructure, but I dropped this for now.
So, for the time being, you get both tracing formats.
Should be easy to add sysctl later on to disable the old format.
include/net/netfilter/nf_tables_core.h | 1 + include/net/netfilter/nft_meta.h | 3 +++ net/bridge/netfilter/nft_meta_bridge.c | 1 + net/netfilter/nf_tables_core.c | 6 +++++- net/netfilter/nf_tables_trace.c | 4 ++++ net/netfilter/nft_meta.c | 16 ++++++++++++++++ 6 files changed, 30 insertions(+), 1 deletion(-)
include/net/netfilter/nf_tables_core.h +1 -0
@@ -57,6 +57,7 @@ struct nft_payload_set { };extern const struct nft_expr_ops nft_payload_fast_ops; +extern struct static_key_false nft_trace_enabled;int nft_payload_module_init (void );void nft_payload_module_exit (void );
include/net/netfilter/nft_meta.h +3 -0
@@ -33,4 +33,7 @@ void nft_meta_set_eval(const struct nft_expr *expr,struct nft_regs *regs, const struct nft_pktinfo *pkt); +void nft_meta_set_destroy (const struct nft_ctx *ctx, +const struct nft_expr *expr); +#endif
net/bridge/netfilter/nft_meta_bridge.c +1 -0
@@ -84,6 +84,7 @@ static const struct nft_expr_ops nft_meta_bridge_set_ops = { .size = NFT_EXPR_SIZE(sizeof(struct nft_meta)), .eval = nft_meta_set_eval, .init = nft_meta_set_init, + .destroy = nft_meta_set_destroy, .dump = nft_meta_set_dump, };
net/netfilter/nf_tables_core.c +5 -1
@@ -16,6 +16,7 @@#include <linux/skbuff.h> #include <linux/netlink.h> #include <linux/netfilter.h> +#include <linux/static_key.h> #include <linux/netfilter/nfnetlink.h> #include <linux/netfilter/nf_tables.h> #include <net/netfilter/nf_tables_core.h>
@@ -62,7 +63,10 @@ static inline void nft_trace_packet(struct nft_traceinfo *info, int rulenum, enum nft_trace_types type) { - if (unlikely(pkt->skb->nf_trace)) { + if (static_branch_unlikely(&nft_trace_enabled)) { +if (!pkt->skb->nf_trace) +return ; + info->chain = chain; info->rule = rule; info->verdict = verdict;
net/netfilter/nf_tables_trace.c +4 -0
@@ -8,6 +8,7 @@*/ #include <linux/module.h> +#include <linux/static_key.h> #include <linux/hash.h> #include <linux/if_vlan.h> #include <linux/init.h>
@@ -25,6 +26,9 @@#define NFT_ID_SKB_HASHBITS 12 +DEFINE_STATIC_KEY_FALSE (nft_trace_enabled); +EXPORT_SYMBOL_GPL (nft_trace_enabled); +/* only used if tracing is on */ static DEFINE_PER_CPU_READ_MOSTLY(u32, trace_seq);static atomic_t trace_cnt;
net/netfilter/nft_meta.c +16 -0
@@ -18,10 +18,12 @@#include <linux/ip.h> #include <linux/ipv6.h> #include <linux/smp.h> +#include <linux/static_key.h> #include <net/dst.h> #include <net/sock.h> #include <net/tcp_states.h> /* for TCP_TIME_WAIT */ #include <net/netfilter/nf_tables.h> +#include <net/netfilter/nf_tables_core.h> #include <net/netfilter/nft_meta.h> void nft_meta_get_eval(const struct nft_expr *expr,
@@ -299,6 +301,9 @@ int nft_meta_set_init(const struct nft_ctx *ctx,if (err <0 )return err; +if (priv->key ==NFT_META_NFTRACE ) +static_branch_inc (&nft_trace_enabled); +return 0 ; }EXPORT_SYMBOL_GPL (nft_meta_set_init);
@@ -336,6 +341,16 @@ nla_put_failure: }EXPORT_SYMBOL_GPL (nft_meta_set_dump); +void nft_meta_set_destroy (const struct nft_ctx *ctx, +const struct nft_expr *expr) +{ +const struct nft_meta *priv =nft_expr_priv (expr); + +if (priv->key ==NFT_META_NFTRACE ) +static_branch_dec (&nft_trace_enabled); +} +EXPORT_SYMBOL_GPL (nft_meta_set_destroy); +static struct nft_expr_type nft_meta_type;static const struct nft_expr_ops nft_meta_get_ops = { .type = &nft_meta_type,
@@ -350,6 +365,7 @@ static const struct nft_expr_ops nft_meta_set_ops = { .size = NFT_EXPR_SIZE(sizeof(struct nft_meta)), .eval = nft_meta_set_eval, .init = nft_meta_set_init, + .destroy = nft_meta_set_destroy, .dump = nft_meta_set_dump, };
Signature · 1 line
2.4.10
Re: [PATCH v2 nf-next 3/3] netfilter: nf_tables: wrap tracing with a static key
Quoted 10 lines · Florian Westphal
Only needed when meta nftrace rule(s) were added.
Signed-off-by Florian Westphal <[email protected]>---
Changes since v1:
- do not wrap rulenum++ into static key, because it might get incremented too late when trace rules are added/removed at high rate, giving wrong results.
- addEXPORT_SYMBOLfornft_meta_set_destroy.
Quoted 7 lines
There was another followup patch to disable old tracing infrastructure, but I dropped this for now.
So, for the time being, you get both tracing formats.
Should be easy to add sysctl later on to disable the old format.
Just an observation so far: this really bloats nft_do_chain(), in my compilation it adds 655 bytes and inlines all the tracing stuff multiple times into critical spots. We really need to get this down.
First simple and easy attempt was the attached patch, it brings it down to only +236b. But I think we can still do better.
One idea would be to wrap the local state in a struct. That way we can pass all of it together using a single pointer. For that We could use the current jump stack frame to hold also the *current* chain and rule pointers if we can make sure the compiler will still use registers for the current ones.
The nft_trace_notify() invocation should probably be moved to __nft_trace_packet() and if still necessary that should be marked noinline.
Just a couple of ideas for a start. We really need to avoid bloating these critical loops.
net/netfilter/nf_tables_core.c +6 -6
@@ -45,14 +45,14 @@ static struct nf_loginfo trace_loginfo = { }, }; -static void __nft_trace_packet (const struct nft_pktinfo *pkt, -const struct nft_chain *chain, -int rulenum,enum nft_trace type) +static noinline void __nft_trace_packet(struct nft_traceinfo *info, +const struct nft_pktinfo *pkt, +int rulenum) {nf_log_trace (pkt->net, pkt->pf, pkt->hook, pkt->skb, pkt->in, pkt->out, &trace_loginfo,"TRACE: %s:%s:%s:%u " , - chain->table->name, chain->name, comments[type], - rulenum); + info->chain->table->name, info->chain->name, + comments[info->type], rulenum); } static inline void nft_trace_packet(struct nft_traceinfo *info,
@@ -73,7 +73,7 @@ static inline void nft_trace_packet(struct nft_traceinfo *info, info->type = type;nft_trace_notify (info, pkt); -__nft_trace_packet (pkt, chain, rulenum, type); +__nft_trace_packet (info, pkt, rulenum); } }
Quoted 7 lines · Florian Westphal
This is V2 of the nft trace patch set.
See the individual patches for a change vs. v1 description.
Patrick: I did a quick test w. your last public patches (plus a few minor changes to make it build again) and things seem to work fine.
Thanks, I have it running, will start making the missing adaptions now.
Quoted 3 lines
What does not work correctly is decoding of vlan header. I think whats happening is that we cannot splice sub-byte sized quantities from a larger chunk.
It does work for IPv4 (version/hdrlength), but they don't cross byte boundaries, maybe its related to that. I'll check it out.
Quoted 1 line
Other than that, things appear to work correctly.
Very nice, thanks!
Quoted 11 lines · Florian Westphal
This is V2 of the nft trace patch set.
See the individual patches for a change vs. v1 description.
Patrick: I did a quick test w. your last public patches (plus a few minor changes to make it build again) and things seem to work fine.
What does not work correctly is decoding of vlan header. I think whats happening is that we cannot splice sub-byte sized quantities from a larger chunk.
I've managed to decode it properly, so the kernel part is fine:
ether saddr 63:f6:4b:00:54:52 ether daddr c9:4b:a9:00:54:52 vlan pcp 0 vlan cfi 0 vlan id 1000 vlan type ip ip saddr 10.0.0.1 ip daddr 10.0.0.2 ip tos 0 ip ttl 64 ip id 61847 ip length 84 icmp type echo-request icmp code 0 icmp id 29049 icmp sequence 4267 icmp type 61 icmp code 177 icmp id 0 icmp sequence 0 icmp type 116 icmp code 211
There are a number of problems however. The easy one is wrong header definitions in nft, also my decoding is a bit hackish so far and occasionally causes endless loops, I'll fix that.
The somewhat bigger problem are some inconsistencies. Usually we can see the original lower layer header on the receive path in the higher layers, f.i. the ethernet header. With VLAN offloading this doesn't work since we loose the meta data once the packet has been received through the VLAN code. This means we'll just see a regular ethernet header.
This does not only affect tracing, but anything that wants to access that header. Its not a *huge* problem, but it is unexpected for the user, especially since he is able to treat the offloaded header like a regular VLAN header before "decapsulation".
I have no good suggestion what to do about that. The information in the skb is gone. The only possibility I can think of is to reconstruct (most of) it based on the incoming device, the known upper protocol and skb->priority, but that's quite ugly.