Skip to content

iptables: introduce cache evaluation phase

Pablo Neira Ayuso netfilter-devel 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest7 patches 0 repliesNew

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7nft: do not check for existing chain from parseriptables/tests/shell/testcases/ip6tables/0004-return-codes_0 iptables/tests/shell/testcases/iptables/0004-return-codes_0 iptables/xtables.c+2 -70 replies
2/7nft: split parsing from netlink commandsiptables/Makefile.am iptables/nft-arp.c iptables/nft-bridge.c +10 more+702 -1490 replies
3/7nft: calculate cache requirements from list of commandsiptables/nft-cache.c iptables/nft-cache.h iptables/nft-cmd.c +5 more+122 -390 replies
4/7nft: restore among supportiptables/nft-bridge.c iptables/nft.c iptables/nft.h+32 -20 replies
5/7nft: remove cache build callsiptables/nft-cache.c iptables/nft.c+0 -270 replies
6/7nft: skip table list release if uninitializediptables/nft-cache.c+4 -20 replies
7/7nft: missing nft_fini() call in bridge familyiptables/xtables-eb-standalone.c+2 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/7] iptables: introduce cache evaluation phase

Hi Phil,

Happy new year.

This patchset introduces a new step to parse rules into a list of commands:

input -> parser -> list of commands -> list of jobs -> netlink -> kernel

This allows us to estimate the cache requirements from the list of commands and to avoid in-transit cache cancelations while handling a batch.

This batch is likely disabling your optimizations where selective chain/set netlink dump speeds up things. From reading the code, I'm not sure what paths are specifically benefiting from this optimization since any of the existing nftnl_chain_list_get() and nftnl_set_list_get() calls might exercise this selective netlink dump. I think it should not be too hard to restore those, I remember you have a few tests to evaluate the speed up. Note that this patch also disables the pre-parsing from xtables-restore.

Downside is that there's an extra memory allocation, that could be consolidated in the future, by having one single list of commands/jobs. But would probably make this batch even larger, but I tried to reduce complexity as much as possible. Debugging cache issue is hard in my opinion, if we can avoid fetch/cancel/fetch cache scenario, the better.

The approach that 'among' follows to use the set infrastructure is interesting, probably there's a way to simplify this, but I understand there might be more urgent stuff to have a look.

Let me know,
Thanks.

P.S: I started this patchset by Dec 25 while traveling, tests look
OK here, I'm just releasing this because I'm not sure I can come back to this anytime soon.

Pablo Neira Ayuso (7):
nft: do not check for existing chain from parser
nft: split parsing from netlink commands
nft: calculate cache requirements from list of commands
nft: restore among support
nft: remove cache build calls
nft: skip table list release if uninitialized
nft: missing nft_fini() call in bridge family

 iptables/Makefile.am                               |   2 +-
 iptables/nft-arp.c                                 |   5 +-
 iptables/nft-bridge.c                              |  18 +-
 iptables/nft-cache.c                               |  72 ++--
 iptables/nft-cache.h                               |   1 +
 iptables/nft-cmd.c                                 | 389 +++++++++++++++++++++
 iptables/nft-cmd.h                                 |  79 +++++
 iptables/nft-shared.c                              |   6 +-
 iptables/nft-shared.h                              |   4 +-
 iptables/nft.c                                     | 310 +++++++++++-----
 iptables/nft.h                                     |  49 ++-
 .../shell/testcases/ip6tables/0004-return-codes_0  |   2 +-
 .../shell/testcases/iptables/0004-return-codes_0   |   2 +-
 iptables/xtables-arp.c                             |  26 +-
 iptables/xtables-eb-standalone.c                   |   2 +
 iptables/xtables-eb.c                              |  26 +-
 iptables/xtables-restore.c                         |  35 +-
 iptables/xtables-save.c                            |   3 +
 iptables/xtables.c                                 |  57 ++-
 19 files changed, 863 insertions(+), 225 deletions(-)
 create mode 100644 iptables/nft-cmd.c
 create mode 100644 iptables/nft-cmd.h
Signature · 1 line
2.11.0