Skip to contentSkip to conversation

netfilter: nf_tables: remove element flush allocation

RFC In review Florian Westphal 2 participants 3 messages Jul 31, 2025 – Aug 7, 2025

Subsystems (2)
lore
Share & tools
lore

Florian Westphal

author maintainer
#0 lore raw

RFC 1/1 netfilter: nf_tables: remove element flush allocation

Quoting Sven Auhagen:
we do see on occasions that we get the following error message, more so on x86 systems than on arm64:

Error: Could not process rule: Cannot allocate memory delete table inet filter

It is not a consistent error and does not happen all the time. We are on Kernel 6.6.80, seems to me like we have something along the lines of the nf_tables: allow clone callbacks to sleep problem using GFP_ATOMIC.

As hinted at by Sven, this is because of GFP_ATOMIC allocations during set flush.

When set is flushed, all elements are deactivated. This triggers a set walk and each element gets added to the transaction list.

This is needed, because the commit phase will unlink (remove) the elements from the underlying data structure (hashtable, tree, ...), but we must delay the freeing until after a synchromize_rcu().

The nft_trans_elem structure holds the addresses of the set elements so we can free them after removal from the set.

One way to resolve this is to allow sleeping allocations, but Pablo suggested to avoid the per-element-allocations altogether.

The main drawback vs the initial patch is that in order to support sleeping allocations, memory cost of each set element grows by one pointer whereas initial sleeping-allocations only did this for the rhashtable backend.

Not signed off as I don't see this as more elegant as v1 here:
https://lore.kernel.org/netfilter-devel/[email protected]/

One advantage however is that NEWSETELEM could be converted to use the llist too instead of the dynamically-sized nelems array.

Then, the array could be removed again, it seems dubious to keep it just for the update case.
That in turn would allow to remove the compaction code again.

Both DEL/NEWSETELEM would be changed to first peek the transaction list tail to see if a compatible transaction exists and re-use that instead of allocating a new one.

Pablo, please let me know if you prefer this direction compared to v1. If so, I would also work on removing the trailing dynamically sized array from nft_trans_elem structure in a followup patch.

Thanks!

---
 include/net/netfilter/nf_tables.h |   5 +
 net/netfilter/nf_tables_api.c     | 157 +++++++++++++++++++-----------
 2 files changed, 104 insertions(+), 58 deletions(-)
include/net/netfilter/nf_tables.h +5 -0
index 891e43a01bdc..6668b186e83c 100644
@@ -735,6 +735,7 @@ struct nft_set_ext_tmpl {
  *
  *	@genmask: generation mask, but also flags (see NFT_SET_ELEM_DEAD_BIT)
  *	@offset: offsets of individual extension types
+ *	@trans_list: used during transaction when element is added or removed
  *	@data: beginning of extension data
  *
  *	This structure must be aligned to word size, otherwise atomic bitops
@@ -743,6 +744,7 @@ struct nft_set_ext_tmpl {
 struct nft_set_ext {
 	u8	genmask;
 	u8	offset[NFT_SET_EXT_NUM];
+	struct llist_node trans_list;
 	char	data[];
 } __aligned(BITS_PER_LONG / 8);
 
@@ -775,6 +777,7 @@ static inline void nft_set_ext_init(struct nft_set_ext *ext,
 				    const struct nft_set_ext_tmpl *tmpl)
 {
 	memcpy(ext->offset, tmpl->offset, sizeof(ext->offset));
+	init_llist_node(&ext->trans_list);
 }
 
 static inline bool __nft_set_ext_exists(const struct nft_set_ext *ext, u8 id)
@@ -1777,6 +1780,8 @@ struct nft_trans_one_elem {
 struct nft_trans_elem {
 	struct nft_trans		nft_trans;
 	struct nft_set			*set;
+	struct llist_head		elem_list;
+	unsigned int 			elem_count;
 	bool				bound;
 	unsigned int			nelems;
 	struct nft_trans_one_elem	elems[] __counted_by(nelems);
net/netfilter/nf_tables_api.c +99 -58

265 lines not on this page · raw message

Signature · 1 line
2.49.1
#1 lore raw

Hi Florian,

On Thu, Jul 31, 2025 at 05:43:49PM +0200, Florian Westphal wrote:
[...]

Quoted 10 lines

One way to resolve this is to allow sleeping allocations, but Pablo suggested to avoid the per-element-allocations altogether.

The main drawback vs the initial patch is that in order to support sleeping allocations, memory cost of each set element grows by one pointer whereas initial sleeping-allocations only did this for the rhashtable backend.

Not signed off as I don't see this as more elegant as v1 here:
https://lore.kernel.org/netfilter-devel/[email protected]/

Not very elegant, maybe it is just incomplete.

Quoted 2 lines

One advantage however is that NEWSETELEM could be converted to use the llist too instead of the dynamically-sized nelems array.

Yes.

Quoted 2 lines

Then, the array could be removed again, it seems dubious to keep it just for the update case.

For updates, I think the element would need a scratch area to store the new timeout/expiration until commit phase is reached. For several updates on the same element in a batch.

Quoted 1 line

That in turn would allow to remove the compaction code again.

Yes.

Quoted 3 lines

Both DEL/NEWSETELEM would be changed to first peek the transaction list tail to see if a compatible transaction exists and re-use that instead of allocating a new one.

Right. Would all this provide even more memory savings?

Quoted 3 lines

Pablo, please let me know if you prefer this direction compared to v1. If so, I would also work on removing the trailing dynamically sized array from nft_trans_elem structure in a followup patch.

I don't remember when precisely, but time ago, you mentioned something like "this transaction infrastructure creates myriad of temporary objects". Your dynamic array infrastructure made it better.

Maybe it is time to integrate transaction infrastrcture more tightly into the existing infrastructure, so there is not need to allocate so many ancilliary objects for large sets?

There is a trade-off in all this.

author maintainer
#2 lore raw
Quoted 4 lines · Pablo Neira Ayuso
Quoted 2 lines

Not signed off as I don't see this as more elegant as v1 here:
https://lore.kernel.org/netfilter-devel/[email protected]/

Not very elegant, maybe it is just incomplete.

Its complete (both the sleepable-iter linked above and this RFC). The RFC omits conversion of NETSETELEM however.
I did not spend time on that because I'm not sure I'm following your suggestion in the first place.

Quoted 5 lines
Quoted 3 lines

Both DEL/NEWSETELEM would be changed to first peek the transaction list tail to see if a compatible transaction exists and re-use that instead of allocating a new one.

Right. Would all this provide even more memory savings?

Yes. The memory saving would come from no-need for elems[], except for update case.

Atm we can store 124 elements in one nft_trans structure. Each nft_trans_elem has 2k size, to hold the pointers to the elements added/removed.

But with list based approach you need one nft_trans struct only (96 byte) in ideal conditions (same op on same set, e.g. flush case).

So its in the 96 byte vs. 163840 bytes range for 10.000 elem del/add.

The downside is the permanent 8-byte per element increase. But, the truckload of temporary trans objects will be gone.

Quoted 8 lines
Quoted 3 lines

Pablo, please let me know if you prefer this direction compared to v1. If so, I would also work on removing the trailing dynamically sized array from nft_trans_elem structure in a followup patch.

I don't remember when precisely, but time ago, you mentioned something like "this transaction infrastructure creates myriad of temporary objects". Your dynamic array infrastructure made it better.

Quoted 5 lines

Maybe it is time to integrate transaction infrastrcture more tightly into the existing infrastructure, so there is not need to allocate so many ancilliary objects for large sets?

There is a trade-off in all this.

Yes, there is. If you agree I will try to extend the RFC patchset:
- add one patch to convert NEWSETELEM case
- add one patch to get rid of elems[].
Unless you think there is a use case for update from userspace that makes a mass update of a set, such as modifying the timeout of 1k elements, then it would be better to keep elems[] and use it only for update case. Let me know what you think -- I don't think its a scenario worth optimizing for.