Skip to content

RFC netfilter: ipset: convert to rhashtable

Florian Westphal netfilter-devel syzbot 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest12 patches 15 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/12netfilter: ipset: rework cidr bookkeepingnet/netfilter/ipset/ip_set_hash_gen.h net/netfilter/ipset/ip_set_hash_ipportnet.c net/netfilter/ipset/ip_set_hash_net.c +4 more+177 -930 replies
2/12netfilter: ipset: rework cidr bookkeeping fixupsnet/netfilter/ipset/ip_set_hash_gen.h+11 -50 replies
3/12netfilter: ipset: add small wrappers for hash and bucket sizesnet/netfilter/ipset/ip_set_hash_gen.h+31 -80 replies
4/12netfilter: ipset: add and use mtype_del_cidr_all helpernet/netfilter/ipset/ip_set_hash_gen.h+16 -180 replies
5/12netfilter: ipset: add and use ip_set_init_comment_slowinclude/linux/netfilter/ipset/ip_set.h net/netfilter/ipset/ip_set_bitmap_gen.h net/netfilter/ipset/ip_set_core.c +2 more+18 -20 replies
6/12netfilter: ipset: add and use ip_set_ext_destroy_slowinclude/linux/netfilter/ipset/ip_set.h net/netfilter/ipset/ip_set_bitmap_gen.h net/netfilter/ipset/ip_set_hash_gen.h+26 -80 replies
7/12netfilter: ipset: add rhashtable boilerplate stubsnet/netfilter/ipset/ip_set_hash_gen.h+102 -22 replies
8/12netfilter: ipset: replace internal hash table with rhashtablenet/netfilter/ipset/ip_set_hash_gen.h+259 -9932 replies
9/12netfilter: ipset: use plain rcu_read_locknet/netfilter/ipset/ip_set_core.c net/netfilter/ipset/ip_set_hash_gen.h net/netfilter/ipset/ip_set_hash_netnet.c +1 more+22 -252 replies
10/12netfilter: ipset: use correct lockdep annotation in ipset_dereferenceinclude/linux/netfilter/ipset/ip_set.h net/netfilter/ipset/ip_set_core.c net/netfilter/ipset/ip_set_hash_gen.h +1 more+8 -62 replies
11/12netfilter: ipset: remove last region lock usageinclude/linux/netfilter/ipset/ip_set.h net/netfilter/ipset/ip_set_bitmap_gen.h net/netfilter/ipset/ip_set_bitmap_ip.c +5 more+36 -424 replies
12/12netfilter: ipset: re-add forceadd support for rhashtablenet/netfilter/ipset/ip_set_hash_gen.h+38 -51 reply

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH RFC nf-next 00/12] netfilter: ipset: convert to rhashtable

Hi,

This is an initial RFC patchset to convert the hash types to rhashtable. Main conversion is in patch 8. First patches contain small drive-by fixes, patches after 8 contain further simplifications/cleanups. Last patch adds back the FORCEADD support dropped in the conversion commit (diff was getting too large...).

The next step is to go through the ipset test failures and figure out which ones hint at actual bugs and which ones are just harmless cosmetic issues (that could be suppressed by tinkering with diff.sh in ipset tests).

Notable likely valid remaining bugs:
- iphash: IP: Compare sorted save and restore: [..] FAILED
- hash:net,iface.t: Check 10.0.1.1 with eth0:
Failed test: ../src/ipset [..] 10.0.1.1,eth0
Warning: 10.0.1.1,eth0 is in set test.

I'd like to eventually get rid of more set->lock places, remove all usage of rcu_dereference_protected(.. , 1), but thats not too urgent atm.

1) Rework ipset CIDR bookkeeping. I had to remove this one from the last nf batch at the last minute because of a buildbot report. See next patch.

2) Fix a few nits in patch 1), to be squash-merged.

3) Add small wrappers for hash and bucket sizes to reduce noise in the actual conversion patch.

4) add and use tmtype_del_cidr_all helper to simplify the upcoming rewrite.

5) Use ip_set_init_comment_slow to prevent race conditions in hash ipset types. Add lockdep annotations.

6) Same as 5, but for remove: adds ip_set_ext_destroy_slow.

7) Add rhashtable boilerplate stubs to ipset. Initialize and destroy the rhashtable without ever adding elements.

8) Replace ipset's internal hash table with rhashtable. FORCEADD is removed here, and added back in last patch.

9) Use plain rcu_read_lock, not _bh variants.

10) Better lockdep annotations in ipset_dereference. Add assertions to more places.

11) Remove the last region lock usage in ipset. Move lock responsibility to kadt, uadt, and flush callbacks.

12) Re-add forceadd support for rhashtable in ipset. Implement mtype_remove_random() to evict elements when the set is full.

Florian Westphal (11):
netfilter: ipset: rework cidr bookkeeping fixups
netfilter: ipset: add small wrappers for hash and bucket sizes
netfilter: ipset: add and use mtype_del_cidr_all helper
netfilter: ipset: add and use ip_set_init_comment_slow
netfilter: ipset: add and use ip_set_ext_destroy_slow
netfilter: ipset: add rhashtable boilerplate stubs
netfilter: ipset: replace internal hash table with rhashtable
netfilter: ipset: use plain rcu_read_lock
netfilter: ipset: use correct lockdep annotation in ipset_dereference
netfilter: ipset: remove last region lock usage
netfilter: ipset: re-add forceadd support for rhashtable

Jozsef Kadlecsik (1):
netfilter: ipset: rework cidr bookkeeping

 include/linux/netfilter/ipset/ip_set.h       |   44 +-
 net/netfilter/ipset/ip_set_bitmap_gen.h      |    6 +
 net/netfilter/ipset/ip_set_bitmap_ip.c       |   11 +-
 net/netfilter/ipset/ip_set_bitmap_ipmac.c    |    9 +-
 net/netfilter/ipset/ip_set_bitmap_port.c     |   11 +-
 net/netfilter/ipset/ip_set_core.c            |   35 +-
 net/netfilter/ipset/ip_set_hash_gen.h        | 1552 ++++++------------
 net/netfilter/ipset/ip_set_hash_ipportnet.c  |    4 +-
 net/netfilter/ipset/ip_set_hash_net.c        |    4 +-
 net/netfilter/ipset/ip_set_hash_netiface.c   |    4 +-
 net/netfilter/ipset/ip_set_hash_netnet.c     |   12 +-
 net/netfilter/ipset/ip_set_hash_netport.c    |    4 +-
 net/netfilter/ipset/ip_set_hash_netportnet.c |   12 +-
 net/netfilter/ipset/ip_set_list_set.c        |   13 +
 14 files changed, 629 insertions(+), 1092 deletions(-)
Signature · 1 line
2.54.0