netfilter: nf_tables: move set_update_list to nftables per-netns
Subsystems (2)
Pablo Neira Ayuso
maintainerPATCH v2 1/2 netfilter: nf_tables: move set_update_list to nftables per-netns
This list is used to invoke the set .commit and .abort ops for the rbtree and pipapo to run GC on expired elements and replace the current datastructure view by the clone. For the rbtree, this also rebuild the datapath b-search array.
From abort path, remove the set from the update_list if it is already bound to rule, then the rule itself takes care of releasing the set and its elements, otherwise, memleak is possible because set ops .abort only deals with removing the set data structure, not the elements.
This is a preparation patch to call set .commit before processing the transaction list for the rbtree, no functional changes are intended.
---
v2: comestic: s/make move/move/ in patch subject.
include/net/netfilter/nf_tables.h | 1 + net/netfilter/nf_tables_api.c | 49 ++++++++++--------------------- 2 files changed, 16 insertions(+), 34 deletions(-)
include/net/netfilter/nf_tables.h +1 -0
@@ -1949,6 +1949,7 @@ struct nftables_pernet {struct list_head binding_list;struct list_head module_list;struct list_head notify_list; +struct list_head set_update_list;struct mutex commit_mutex;u64 table_handle;u64 tstamp;
net/netfilter/nf_tables_api.c +15 -34
140 lines not on this page · raw message
Signature · 1 line
2.47.3
Pablo Neira Ayuso
maintainerPATCH v2 2/2 netfilter: nf_tables: call set ops .commit when building new ruleset
The rbtree set only builds the b-search array after the new ruleset has been exposed through set ops .commit.
This is currently needed by pipapo because it purges the elements from the clone after the transactions are handled, therefore, pipapo still needs the delayed set ops .commit call after the transaction handling.
Allow the rbtree to call .commit before the transaction handling which purges the stale elements from the frontend rbtree datastructure.
Update rbtree .commit to skip deactivated and expired elements when building the new b-search array.
---
sashiko: there is a timestamp that prevents elements from expiring while handling the transaction.
v2: - use NFT_GENMASK_ANY for GC scan, theoretically no new element in
this transaction can expire while handling the transaction but let's just skip either new element or deactivated elements from the GC scan.
- remove incorrect reset of previous rbe when performing GC scan.
net/netfilter/nf_tables_api.c | 9 ++++++-- net/netfilter/nft_set_rbtree.c | 38 ++++++++++++++++++++++++++++------ 2 files changed, 39 insertions(+), 8 deletions(-)
net/netfilter/nf_tables_api.c +7 -2
@@ -10853,11 +10853,14 @@ static void nf_tables_commit_audit_log(struct list_head *adl, u32 generation) } } -static void nft_set_commit_update (struct nftables_pernet *nft_net) +static void nft_set_commit_update (struct nftables_pernet *nft_net,bool early_commit) {struct nft_set *set, *next;list_for_each_entry_safe (set, next, &nft_net->set_update_list, pending_update) { +if (set->ops->abort_skip_removal && early_commit) +continue ; +list_del_init (&set->pending_update);if (!set->ops->commit || set->dead)
@@ -10964,6 +10967,8 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb) }/* step 2. Make rules_gen_X visible to packet path */ +nft_set_commit_update (nft_net, true); +list_for_each_entry (table, &nft_net->tables, list) {list_for_each_entry (chain, &table->chains, list)nf_tables_commit_chain (net, chain);
@@ -11170,7 +11175,7 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb) } } -nft_set_commit_update (nft_net); +nft_set_commit_update (nft_net, false);nft_commit_notify (net,NETLINK_CB (skb).portid);nf_tables_gen_notify (net, skb,NFT_MSG_NEWGEN );
net/netfilter/nft_set_rbtree.c +32 -6
@@ -901,6 +901,8 @@ static void nft_rbtree_gc_scan(struct nft_set *set) next =rb_next (node); rbe = rb_entry(node, struct nft_rbtree_elem, node); +if (!nft_set_elem_active (&rbe->ext,NFT_GENMASK_ANY )) +continue ;/* elements are reversed in the rbtree for historical reasons, * from highest to lowest value, that is why end element is
@@ -1036,10 +1038,32 @@ static void nft_array_free_rcu(struct rcu_head *rcu_head)__nft_array_free (array); } +static struct nft_rbtree_elem * +__nft_rbtree_prev_active (struct rb_node **pnode,u8 genmask) +{ +struct nft_rbtree_elem *prev_rbe; +struct rb_node *node = *pnode; + +while (node) { + prev_rbe = rb_entry(node, struct nft_rbtree_elem, node); +if (!nft_set_elem_active (&prev_rbe->ext, genmask)) { + node =rb_prev (node); +continue ; + } + + *pnode = node; +return prev_rbe; + } + +return NULL ; +} +static void nft_rbtree_commit (struct nft_set *set) {struct nft_rbtree *priv =nft_set_priv (set);struct nft_rbtree_elem *rbe, *prev_rbe; +struct net *net =read_pnet (&set->net); +u8 genmask =nft_genmask_next (net);struct nft_array *old;u32 num_intervals =0 ;struct rb_node *node;
@@ -1061,12 +1085,12 @@ static void nft_rbtree_commit(struct nft_set *set)/* Reverse walk to create an array from smaller to largest interval. */ node =rb_last (&priv->root); -if (node) - prev_rbe = rb_entry(node, struct nft_rbtree_elem, node); -else - prev_rbe =NULL ; -while (prev_rbe) { +while (node) { + prev_rbe =__nft_rbtree_prev_active (&node, genmask); +if (!prev_rbe) +break ; + rbe = prev_rbe;if (nft_rbtree_interval_start (rbe))
@@ -1083,7 +1107,9 @@ static void nft_rbtree_commit(struct nft_set *set)if (!node)break ; - prev_rbe = rb_entry(node, struct nft_rbtree_elem, node); + prev_rbe =__nft_rbtree_prev_active (&node, genmask); +if (!prev_rbe) +break ;/* For anonymous sets, when adjacent ranges are found, * the end element is not added to the set to pack the set
Signature · 1 line
2.47.3
Re: [PATCH nf,v2 2/2] netfilter: nf_tables: call set ops .commit when building new ruleset
Quoted 35 lines · Pablo Neira Ayuso
The rbtree set only builds the b-search array after the new ruleset has been exposed through set ops .commit.
This is currently needed by pipapo because it purges the elements from the clone after the transactions are handled, therefore, pipapo still needs the delayed set ops .commit call after the transaction handling.
Allow the rbtree to call .commit before the transaction handling which purges the stale elements from the frontend rbtree datastructure.
Update rbtree .commit to skip deactivated and expired elements when building the new b-search array.
Signed-off-by Pablo Neira Ayuso <[email protected]>---
sashiko: there is a timestamp that prevents elements from expiring while handling the transaction.v2: - use
NFT_GENMASK_ANYfor GC scan, theoretically no new element in
this transaction can expire while handling the transaction but let's just skip either new element or deactivated elements from the GC scan.
- remove incorrect reset of previous rbe when performing GC scan.net/netfilter/nf_tables_api.c | 9 ++++++-- net/netfilter/nft_set_rbtree.c | 38 ++++++++++++++++++++++++++++------ 2 files changed, 39 insertions(+), 8 deletions(-)net/netfilter/nf_tables_api.c +0
-0@@ -10853,11 +10853,14 @@ static void nf_tables_commit_audit_log(struct list_head *adl, u32 generation) } }
net/netfilter/nf_tables_api.c static void nf_tables_commit_audit_log(struct list_head *adl, u32 generation) · line 10863
Quoted 10 lines
net/netfilter/nf_tables_api.c +4
-1-static void nft_set_commit_update (struct nftables_pernet *nft_net) +static void nft_set_commit_update (struct nftables_pernet *nft_net,bool early_commit) {struct nft_set *set, *next;list_for_each_entry_safe (set, next, &nft_net->set_update_list, pending_update) { +if (set->ops->abort_skip_removal && early_commit) +continue ; +
Hi Pablo,
wouldn't it be better to add the early_commit flag to set->ops? This way we don't need to check abort_skip_removal flag.
I could foresee problems in the future using the same flag for this skip..
What about creating a nft_commit_phase enum:
enum nft_commit_phase {
NFT_COMMIT_PHASE_EARLY,
NFT_COMMIT_PHASE_DELAYED,
};
and then we can use it in nft_set_commit_update() as argument, something like:
static void nft_set_commit_update(struct nftables_pernet *nft_net,
enum nft_commit_phase phase)
This way, early_commit makes sense on set->ops..
What do you think? I am trying to think in a way to make the code more readable.
Thanks!
Fernando.
Quoted 96 lines
net/netfilter/nf_tables_api.c +3
-1list_del_init (&set->pending_update);if (!set->ops->commit || set->dead)@@ -10964,6 +10967,8 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb) }/* step 2. Make rules_gen_X visible to packet path */ +nft_set_commit_update (nft_net, true); +list_for_each_entry (table, &nft_net->tables, list) {list_for_each_entry (chain, &table->chains, list)nf_tables_commit_chain (net, chain);@@ -11170,7 +11175,7 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb) } } -nft_set_commit_update (nft_net); +nft_set_commit_update (nft_net, false);nft_commit_notify (net,NETLINK_CB (skb).portid);nf_tables_gen_notify (net, skb,NFT_MSG_NEWGEN );net/netfilter/nft_set_rbtree.c +32
-6@@ -901,6 +901,8 @@ static void nft_rbtree_gc_scan(struct nft_set *set) next =rb_next (node); rbe = rb_entry(node, struct nft_rbtree_elem, node); +if (!nft_set_elem_active (&rbe->ext,NFT_GENMASK_ANY )) +continue ;/* elements are reversed in the rbtree for historical reasons, * from highest to lowest value, that is why end element is @@ -1036,10 +1038,32 @@ static void nft_array_free_rcu(struct rcu_head *rcu_head)__nft_array_free (array); } +static struct nft_rbtree_elem * +__nft_rbtree_prev_active (struct rb_node **pnode,u8 genmask) +{ +struct nft_rbtree_elem *prev_rbe; +struct rb_node *node = *pnode; + +while (node) { + prev_rbe = rb_entry(node, struct nft_rbtree_elem, node); +if (!nft_set_elem_active (&prev_rbe->ext, genmask)) { + node =rb_prev (node); +continue ; + } + + *pnode = node; +return prev_rbe; + } + +return NULL ; +} +static void nft_rbtree_commit (struct nft_set *set) {struct nft_rbtree *priv =nft_set_priv (set);struct nft_rbtree_elem *rbe, *prev_rbe; +struct net *net =read_pnet (&set->net); +u8 genmask =nft_genmask_next (net);struct nft_array *old;u32 num_intervals =0 ;struct rb_node *node;@@ -1061,12 +1085,12 @@ static void nft_rbtree_commit(struct nft_set *set)/* Reverse walk to create an array from smaller to largest interval. */ node =rb_last (&priv->root); -if (node) - prev_rbe = rb_entry(node, struct nft_rbtree_elem, node); -else - prev_rbe =NULL ; -while (prev_rbe) { +while (node) { + prev_rbe =__nft_rbtree_prev_active (&node, genmask); +if (!prev_rbe) +break ; + rbe = prev_rbe;if (nft_rbtree_interval_start (rbe))@@ -1083,7 +1107,9 @@ static void nft_rbtree_commit(struct nft_set *set)if (!node)break ; - prev_rbe = rb_entry(node, struct nft_rbtree_elem, node); + prev_rbe =__nft_rbtree_prev_active (&node, genmask); +if (!prev_rbe) +break ;/* For anonymous sets, when adjacent ranges are found, * the end element is not added to the set to pack the set
Pablo Neira Ayuso
maintainerRe: [PATCH nf,v2 2/2] netfilter: nf_tables: call set ops .commit when building new ruleset
Hi Fernando,
Quoted 62 lines · Fernando Fernandez Mancera
Quoted 33 lines · Pablo Neira Ayuso
The rbtree set only builds the b-search array after the new ruleset has been exposed through set ops .commit.
This is currently needed by pipapo because it purges the elements from the clone after the transactions are handled, therefore, pipapo still needs the delayed set ops .commit call after the transaction handling.
Allow the rbtree to call .commit before the transaction handling which purges the stale elements from the frontend rbtree datastructure.
Update rbtree .commit to skip deactivated and expired elements when building the new b-search array.
Signed-off-by Pablo Neira Ayuso <[email protected]>---
sashiko: there is a timestamp that prevents elements from expiring while handling the transaction.v2: - use
NFT_GENMASK_ANYfor GC scan, theoretically no new element in
this transaction can expire while handling the transaction but let's just skip either new element or deactivated elements from the GC scan.
- remove incorrect reset of previous rbe when performing GC scan.net/netfilter/nf_tables_api.c | 9 ++++++-- net/netfilter/nft_set_rbtree.c | 38 ++++++++++++++++++++++++++++------ 2 files changed, 39 insertions(+), 8 deletions(-)net/netfilter/nf_tables_api.c +0
-0net/netfilter/nf_tables_api.c static void nf_tables_commit_audit_log(struct list_head *adl, u32 generation) · line 10861
Quoted 10 lines
net/netfilter/nf_tables_api.c +4
-1@@ -10853,11 +10853,14 @@ static void nf_tables_commit_audit_log(struct list_head *adl, u32 generation) } } -static void nft_set_commit_update (struct nftables_pernet *nft_net) +static void nft_set_commit_update (struct nftables_pernet *nft_net,bool early_commit) {struct nft_set *set, *next;list_for_each_entry_safe (set, next, &nft_net->set_update_list, pending_update) { +if (set->ops->abort_skip_removal && early_commit) +continue ; +Hi Pablo,
wouldn't it be better to add the
early_commitflag toset->ops? This way we don't need to checkabort_skip_removalflag.I could foresee problems in the future using the same flag for this skip..
What about creating a
nft_commit_phaseenum:enum
nft_commit_phase{
NFT_COMMIT_PHASE_EARLY,
NFT_COMMIT_PHASE_DELAYED,
};and then we can use it in
nft_set_commit_update()as argument, something like:
Quoted 7 lines
static void nft_set_commit_update(struct
nftables_pernet*nft_net,
enumnft_commit_phasephase)This way,
early_commitmakes sense onset->ops..What do you think? I am trying to think in a way to make the code more readable.
The (ab)use of ->abort_skip_removal is hack to address the reported regression with minimal changes.
I can add an .early_commit interface instead, but where I want to go is to move .commit before transaction handling, but pipapo needs to be revisited to make it. Because pipapo has the same issue reported here:
https://lore.kernel.org/netfilter-devel/[email protected]/
The update for pipapo would be targetted at the nf-next.
I can add a .early_commit, that would be cleaner and it is not a lot more code.
Eventually, .early_commit will just become .commit once pipapo becomes safe to be used before transaction handling.
Re: [PATCH nf,v2 2/2] netfilter: nf_tables: call set ops .commit when building new ruleset
Quoted 82 lines · Pablo Neira Ayuso
Hi Fernando,
Quoted 62 lines · Fernando Fernandez Mancera
Quoted 33 lines · Pablo Neira Ayuso
The rbtree set only builds the b-search array after the new ruleset has been exposed through set ops .commit.
This is currently needed by pipapo because it purges the elements from the clone after the transactions are handled, therefore, pipapo still needs the delayed set ops .commit call after the transaction handling.
Allow the rbtree to call .commit before the transaction handling which purges the stale elements from the frontend rbtree datastructure.
Update rbtree .commit to skip deactivated and expired elements when building the new b-search array.
Signed-off-by Pablo Neira Ayuso <[email protected]>---
sashiko: there is a timestamp that prevents elements from expiring while handling the transaction.v2: - use
NFT_GENMASK_ANYfor GC scan, theoretically no new element in
this transaction can expire while handling the transaction but let's just skip either new element or deactivated elements from the GC scan.
- remove incorrect reset of previous rbe when performing GC scan.net/netfilter/nf_tables_api.c | 9 ++++++-- net/netfilter/nft_set_rbtree.c | 38 ++++++++++++++++++++++++++++------ 2 files changed, 39 insertions(+), 8 deletions(-)net/netfilter/nf_tables_api.c +0
-0net/netfilter/nf_tables_api.c static void nf_tables_commit_audit_log(struct list_head *adl, u32 generation) · line 10861
Quoted 10 lines
net/netfilter/nf_tables_api.c +4
-1@@ -10853,11 +10853,14 @@ static void nf_tables_commit_audit_log(struct list_head *adl, u32 generation) } } -static void nft_set_commit_update (struct nftables_pernet *nft_net) +static void nft_set_commit_update (struct nftables_pernet *nft_net,bool early_commit) {struct nft_set *set, *next;list_for_each_entry_safe (set, next, &nft_net->set_update_list, pending_update) { +if (set->ops->abort_skip_removal && early_commit) +continue ; +Hi Pablo,
wouldn't it be better to add the
early_commitflag toset->ops? This way we don't need to checkabort_skip_removalflag.I could foresee problems in the future using the same flag for this skip..
What about creating a
nft_commit_phaseenum:enum
nft_commit_phase{
NFT_COMMIT_PHASE_EARLY,
NFT_COMMIT_PHASE_DELAYED,
};and then we can use it in
nft_set_commit_update()as argument, something like:Quoted 7 lines
static void nft_set_commit_update(struct
nftables_pernet*nft_net,
enumnft_commit_phasephase)This way,
early_commitmakes sense onset->ops..What do you think? I am trying to think in a way to make the code more readable.
The (ab)use of ->abort_skip_removal is hack to address the reported regression with minimal changes.
I can add an .early_commit interface instead, but where I want to go is to move .commit before transaction handling, but pipapo needs to be revisited to make it. Because pipapo has the same issue reported here:
Quoted 8 lines
The update for pipapo would be targetted at the nf-next.
I can add a .early_commit, that would be cleaner and it is not a lot more code.
Eventually, .early_commit will just become .commit once pipapo becomes safe to be used before transaction handling.
Sounds good to me, in essence I would like to avoid re-using a flag that isn't supposed to be used for that. In any case, if this is a temporary change it might be fine too..
Thanks!
Florian Westphal
maintainerRe: [PATCH nf,v2 2/2] netfilter: nf_tables: call set ops .commit when building new ruleset
In reply to Pablo Neira AyusoQuoted 7 lines · Pablo Neira Ayuso
The rbtree set only builds the b-search array after the new ruleset has been exposed through set ops .commit.
This is currently needed by pipapo because it purges the elements from the clone after the transactions are handled, therefore, pipapo still needs the delayed set ops .commit call after the transaction handling.
Quoted 5 lines
Allow the rbtree to call .commit before the transaction handling which purges the stale elements from the frontend rbtree datastructure.
Update rbtree .commit to skip deactivated and expired elements when building the new b-search array.
I'm not following, sorry. What's the difference between pipapo and rbtree?
static void nft_pipapo_commit(struct nft_set *set) {
[..]
if (time_after_eq(jiffies, priv->last_gc + nft_set_gc_interval(set)))
pipapo_gc_scan(set, priv->clone);
old = rcu_replace_pointer(priv->match, priv->clone,
nft_pipapo_transaction_mutex_held(set));[..] after this, new incarnation is live.
static void nft_rbtree_commit(struct nft_set *set) {
[..]
if (time_after_eq(jiffies, priv->last_gc + nft_set_gc_interval(set)))
nft_rbtree_gc_scan(set);[ ... build the new blob ... ]
err_out:
priv->array_next->num_intervals = num_intervals;
old = rcu_replace_pointer(priv->array, priv->array_next,
lockdep_is_held(&nft_pernet(read_pnet(&set->net))->commit_mutex));[..] after this, new incarnation is live.
What is the problem? These two functions do the same thing, no? Why must the blob be rebuilt before stale node purge in rbtree case?
Pablo Neira Ayuso
maintainerRe: [PATCH nf,v2 2/2] netfilter: nf_tables: call set ops .commit when building new ruleset
Quoted 8 lines · Florian Westphal
Quoted 7 lines · Pablo Neira Ayuso
The rbtree set only builds the b-search array after the new ruleset has been exposed through set ops .commit.
This is currently needed by pipapo because it purges the elements from the clone after the transactions are handled, therefore, pipapo still needs the delayed set ops .commit call after the transaction handling.
Quoted 7 lines
Quoted 5 lines
Allow the rbtree to call .commit before the transaction handling which purges the stale elements from the frontend rbtree datastructure.
Update rbtree .commit to skip deactivated and expired elements when building the new b-search array.
I'm not following, sorry. What's the difference between pipapo and rbtree?
pipapo needs to call nft_setelem_remove() to purge the deactivated elements from priv->clone, then the new version of the pipapo lookup table datastructure can be published.
The rbtree does not need to wait to call nft_setelem_remove() since it updates the rbtree frontend datastructure that is only exposed to control plane.
Quoted 20 lines
static void
nft_pipapo_commit(struct nft_set *set){
[..]if (time_after_eq(jiffies, priv->last_gc + nft_set_gc_interval(set))) pipapo_gc_scan(set, priv->clone); old = rcu_replace_pointer(priv->match, priv->clone, nft_pipapo_transaction_mutex_held(set));[..] after this, new incarnation is live.
static void
nft_rbtree_commit(struct nft_set *set){
[..]if (time_after_eq(jiffies, priv->last_gc + nft_set_gc_interval(set))) nft_rbtree_gc_scan(set);[ ... build the new blob ... ]
err_out:
Quoted 8 lines
priv->array_next->num_intervals = num_intervals; old = rcu_replace_pointer(priv->array, priv->array_next, lockdep_is_held(&nft_pernet(read_pnet(&set->net))->commit_mutex));[..] after this, new incarnation is live.
What is the problem? These two functions do the same thing, no?
Apparently yes, but...
Quoted 1 line
Why must the blob be rebuilt before stale node purge in rbtree case?
... there is a gap between the ruleset blob is built and published and the set .commit interface is called to publish the new version of the rbtree/pipapo datastructure.
See:
https://lore.kernel.org/netfilter-devel/[email protected]/
Florian Westphal
maintainerRe: [PATCH nf,v2 2/2] netfilter: nf_tables: call set ops .commit when building new ruleset
Quoted 8 lines · Pablo Neira Ayuso
Quoted 1 line
Why must the blob be rebuilt before stale node purge in rbtree case?
... there is a gap between the ruleset blob is built and published and the set .commit interface is called to publish the new version of the rbtree/pipapo datastructure.
See:
https://lore.kernel.org/netfilter-devel/[email protected]/
Ah. That makes sense. So the problem is not rbtree specific. Problem is that packets switch over to the *new base chain* (linked to nf machinery, nft rule blob becomes reachable on base_seq swap:
1. commit phase starts.
2. seqcount gets bumped.
A. Packet p1 enters machinery
3. elements get purged / chains / tables unlinked, netlink notificatons etc. etc.
B. Packet p1 is in nft_lookup, which gets updated base_seq,
but no match because rbtree blob resp. pipapo live blob are empty in the 'flush ruleset; table t { ..' case.
4. nft_set_commit_update() is called.
If the above is right your patch makes much more sense now :-)
The logic with (set->ops->abort_skip_removal && early_commit) however is hard to grasp.
Even with NFT_COMMIT_PHASE_EARLY or whatever its bad because the set implementation details leak into the transaction phase. But I understand that you'd like to at least solve it for rbtree with a smaller change, so thats ok.
Is there a long-term plan?
Maybe your 'pre-commit' phase could iterate the transaction log and relink transactions (add/del/update of set elements ) to the owning set? Contradicting transactions (destroy set x, then remove element from x) should have been caught earlier, so this delete-from-transaction-log-and-link-to-per-set-struct should not be a problem. Perhaps we might see issues with changes in the netlink reporting order... but thats hopefully easy to avoid.
the ->commit() callback could then access the pending transactions for the set (element adds/deletes) and always get invoked early.
Pipapo could walk its specific elem deletions internally, then swap.
The only other issue I see is that we need a second list_for_each_entry_safe(trans, next, &nft_net->commit_list, list)
... walk, because the step-1 walk is allowed to fail.
Pablo Neira Ayuso
maintainerRe: [PATCH nf,v2 2/2] netfilter: nf_tables: call set ops .commit when building new ruleset
Hi Florian,
Quoted 14 lines · Florian Westphal
Quoted 8 lines · Pablo Neira Ayuso
Quoted 1 line
Why must the blob be rebuilt before stale node purge in rbtree case?
... there is a gap between the ruleset blob is built and published and the set .commit interface is called to publish the new version of the rbtree/pipapo datastructure.
See:
https://lore.kernel.org/netfilter-devel/[email protected]/Ah. That makes sense. So the problem is not rbtree specific. Problem is that packets switch over to the *new base chain* (linked to nf machinery, nft rule blob becomes reachable on
base_seqswap:
Quoted 9 lines
1. commit phase starts.
2. seqcount gets bumped.
A. Packet p1 enters machinery
3. elements get purged / chains / tables unlinked, netlink notificatons etc. etc.
B. Packet p1 is innft_lookup, which gets updatedbase_seq,
but no match because rbtree blob resp. pipapo live blob are empty in the 'flush ruleset; table t { ..' case.
4.nft_set_commit_update()is called.
Yes. The empty set is exposed to the packet path for a short time span, until nft_set_commit_update() is called.
Quoted 4 lines
If the above is right your patch makes much more sense now :-)
The logic with (
set->ops->abort_skip_removal&&early_commit) however is hard to grasp.
I can add a specific .early_commit in v3 as Fernando suggested.
Quoted 18 lines
Even with
NFT_COMMIT_PHASE_EARLYor whatever its bad because the set implementation details leak into the transaction phase. But I understand that you'd like to at least solve it for rbtree with a smaller change, so thats ok.Is there a long-term plan?
Maybe your 'pre-commit' phase could iterate the transaction log and relink transactions (add/del/update of set elements ) to the owning set? Contradicting transactions (destroy set x, then remove element from x) should have been caught earlier, so this delete-from-transaction-log-and-link-to-per-set-struct should not be a problem. Perhaps we might see issues with changes in the netlink reporting order... but thats hopefully easy to avoid.
the ->commit() callback could then access the pending transactions for the set (element adds/deletes) and always get invoked early.
Quoted 6 lines
Pipapo could walk its specific elem deletions internally, then swap.
The only other issue I see is that we need a second
list_for_each_entry_safe(trans, next, &nft_net->commit_list, list)... walk, because the step-1 walk is allowed to fail.
I was considering to address this from the pipapo datastructure itself. The idea would be similar to what the rbtree does, which is to offer a front-end pipapo datastructure that is exposed to control plane that can be updated under the commit mutex and that is not exposed to packet path. Then use it from this new .early_commit interface to clone it, then publish the new version and release the stale via RCU.
Same approach for the bitmap set.
As for the non-rhashtable version, it could be possible to build a compact hashtable based on the control plane datastructure, ie. rhashtable would be only used for control plane and a compact hashtable (using array for buckets and chains) will be built from the new .early_commit interface to publish to datapath.
Exception are dynsets, where a single rhashtable can be used, both to control and datapath.