iptables: introduce cache evaluation phase
Phil Sutter
reviewerv2 cover iptables: introduce cache evaluation phase
Hi Pablo,
As promised, here's a revised version of your cache rework series from January. It restores performance according to my tests (which are yet to be published somewhere) and passes the testsuites.
Patches 1-3 are code simplifications which are not directly related to the actual caching changes.
Patch 4 enhances set fetching by adding support for passing a table name to kernel but no set name. Not a big deal for iptables but it aligns the code with chain fetching.
Patch 5 is a respin of a patch submitted a few weeks ago, namely adding implicit commits to arptables- and ebtables-restore tools which don't support explicit COMMIT lines in input. Big benefit here is that we won't see consecutive commands for different tables anymore, so selective cache fetching doesn't have to deal with too many odd cases.
Patches 6-10 are yours, I rebased and revisited them. Any changes are recorded in per-patch changelogs.
Patch 11 simplifies fetch_set_cache() and fetch_rule_cache() functions as they no longer have to be aware of previous invocations.
Patch 12 improves NFT_CL_FAKE integration considerably, easily possible now that there is nft_cache_level_set() function.
Patch 13 introduces an embedded struct into struct nft_handle which holds cache requirements collected from parsed commands. At first there is just the desired cache level, further patches extend it.
Patches 14-16 re-establish per table/chain cache.
Patch 17 reduces cache requirements for flush command by making nft_xt_builtin_init() cache-aware.
Patch 18 fixes the segfault reported in nfbz#1407.
Pablo Neira Ayuso (5):
nft: split parsing from netlink commands
nft: calculate cache requirements from list of commands
nft: restore among support
nft: remove cache build calls
nft: missing nft_fini() call in bridge family
Phil Sutter (13):
ebtables-restore: Drop custom table flush routine
nft: cache: Eliminate init_chain_cache()
nft: cache: Init per table set list along with chain list
nft: cache: Fetch sets per table
ebtables-restore: Table line to trigger implicit commit
nft: cache: Simplify rule and set fetchers
nft: cache: Improve fake cache integration
nft: cache: Introduce struct nft_cache_req
nft-cache: Fetch cache per table
nft-cache: Introduce __fetch_chain_cache()
nft: cache: Fetch cache for specific chains
nft: cache: Optimize caching for flush command
nft: Fix for '-F' in iptables dumps
iptables/Makefile.am | 2 +- iptables/nft-arp.c | 5 +- iptables/nft-bridge.c | 18 +- iptables/nft-cache.c | 318 +++++++------- iptables/nft-cache.h | 6 +- iptables/nft-cmd.c | 387 ++++++++++++++++++ iptables/nft-cmd.h | 79 ++++ iptables/nft-shared.c | 6 +- iptables/nft-shared.h | 4 +- iptables/nft.c | 369 ++++++++++++----- iptables/nft.h | 62 ++- .../testcases/ip6tables/0004-return-codes_0 | 1 + .../testcases/iptables/0004-return-codes_0 | 6 + .../testcases/nft-only/0006-policy-override_0 | 29 ++ iptables/xtables-arp.c | 26 +- iptables/xtables-eb-standalone.c | 2 + iptables/xtables-eb.c | 26 +- iptables/xtables-restore.c | 126 +----- iptables/xtables-save.c | 3 + iptables/xtables.c | 57 ++- 20 files changed, 1100 insertions(+), 432 deletions(-) create mode 100644 iptables/nft-cmd.c create mode 100644 iptables/nft-cmd.h create mode 100755 iptables/tests/shell/testcases/nft-only/0006-policy-override_0
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 1/18 ebtables-restore: Drop custom table flush routine
At least since flushing xtables-restore doesn't fetch chains from kernel anymore, problems with pending policy rule delete jobs can't happen anymore.
--- iptables/nft.c | 21 --------------------- iptables/nft.h | 1 - iptables/xtables-restore.c | 9 +-------- 3 files changed, 1 insertion(+), 30 deletions(-)
iptables/nft.c +0 -21
@@ -2985,27 +2985,6 @@ int nft_abort(struct nft_handle *h)return nft_action (h,NFT_COMPAT_ABORT ); } -int nft_abort_policy_rule (struct nft_handle *h,const char *table) -{ -struct obj_update *n, *tmp; - -list_for_each_entry_safe (n, tmp, &h->obj_list, head) { -if (n->type !=NFT_COMPAT_RULE_APPEND && - n->type !=NFT_COMPAT_RULE_DELETE ) -continue ; - -if (strcmp (table, -nftnl_rule_get_str (n->rule,NFTNL_RULE_TABLE ))) -continue ; - -if (!nft_rule_is_policy_rule (n->rule)) -continue ; - -batch_obj_del (h, n); - } -return 0 ; -} -int nft_compatible_revision (const char *name,uint8_t rev,int opt) {struct mnl_socket *nl;
iptables/nft.h +0 -1
@@ -160,7 +160,6 @@ uint32_t nft_invflags2cmp(uint32_t invflags, uint32_t flag);int nft_commit (struct nft_handle *h);int nft_bridge_commit (struct nft_handle *h);int nft_abort (struct nft_handle *h); -int nft_abort_policy_rule (struct nft_handle *h,const char *table);/* * revision compatibility.
iptables/xtables-restore.c +1 -8
@@ -484,17 +484,10 @@ int xtables_ip6_restore_main(int argc, char *argv[]) argc, argv); } -static int ebt_table_flush (struct nft_handle *h,const char *table) -{ -/* drop any pending policy rule add/removal jobs */ -nft_abort_policy_rule (h, table); -return nft_table_flush (h, table); -} -static const struct nft_xt_restore_cb ebt_restore_cb = { .commit = nft_bridge_commit, .table_new = nft_table_new, - .table_flush = ebt_table_flush, + .table_flush = nft_table_flush, .do_command = do_commandeb, .chain_set = nft_chain_set, .chain_restore = nft_chain_restore,
Signature · 1 line
2.25.1
Re: [iptables PATCH v2 01/18] ebtables-restore: Drop custom table flush routine
Quoted 3 lines · Phil Sutter
At least since flushing xtables-restore doesn't fetch chains from kernel anymore, problems with pending policy rule delete jobs can't happen anymore.
Phil Sutter
reviewerPATCH v2 2/18 nft: cache: Eliminate init_chain_cache()
In reply to Phil SutterThe function is always called immediately after fetch_table_cache(), so merge it into the latter.
--- iptables/nft-cache.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-)
iptables/nft-cache.c +4 -12
@@ -86,7 +86,7 @@ static int fetch_table_cache(struct nft_handle *h)char buf[16536 ];struct nlmsghdr *nlh;struct nftnl_table_list *list; -int ret; +int i, ret;if (h->cache->tables)return 0 ;
@@ -104,13 +104,6 @@ static int fetch_table_cache(struct nft_handle *h) h->cache->tables = list; -return 1 ; -} - -static int init_chain_cache (struct nft_handle *h) -{ -int i; -for (i =0 ; i <NFT_TABLE_MAX ; i++) {enum nft_table_type type = h->tables[i].type;
@@ -119,9 +112,10 @@ static int init_chain_cache(struct nft_handle *h) h->cache->table[type].chains =nftnl_chain_list_alloc ();if (!h->cache->table[type].chains) -return -1 ; +return 0 ; } -return 0 ; + +return 1 ; }struct nftnl_chain_list_cb_data {
@@ -458,7 +452,6 @@ __nft_build_cache(struct nft_handle *h, enum nft_cache_level level,switch (h->cache_level) {case NFT_CL_NONE :fetch_table_cache (h); -init_chain_cache (h);if (level ==NFT_CL_TABLES )break ;/* fall through */
@@ -505,7 +498,6 @@ void nft_build_cache(struct nft_handle *h, struct nftnl_chain *c)void nft_fake_cache (struct nft_handle *h) {fetch_table_cache (h); -init_chain_cache (h); h->cache_level =NFT_CL_FAKE ;mnl_genid_get (h, &h->nft_genid);
Signature · 1 line
2.25.1
Re: [iptables PATCH v2 02/18] nft: cache: Eliminate init_chain_cache()
Quoted 2 lines · Phil Sutter
The function is always called immediately after
fetch_table_cache(), so merge it into the latter.
Yes, function is also small enough, so:
Phil Sutter
reviewerPATCH v2 3/18 nft: cache: Init per table set list along with chain list
In reply to Phil SutterThis simplifies code a bit and also aligns set and chain lists handling in cache.
--- iptables/nft-cache.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-)
iptables/nft-cache.c +4 -15
@@ -113,6 +113,10 @@ static int fetch_table_cache(struct nft_handle *h) h->cache->table[type].chains =nftnl_chain_list_alloc ();if (!h->cache->table[type].chains)return 0 ; + + h->cache->table[type].sets =nftnl_set_list_alloc (); +if (!h->cache->table[type].sets) +return 0 ; }return 1 ;
@@ -254,21 +258,6 @@ static int fetch_set_cache(struct nft_handle *h,char buf[16536 ];int i, ret; -if (!t) { -for (i =0 ; i <NFT_TABLE_MAX ; i++) { -enum nft_table_type type = h->tables[i].type; - -if (!h->tables[i].name) -continue ; - - h->cache->table[type].sets =nftnl_set_list_alloc (); -if (!h->cache->table[type].sets) -return -1 ; - } - }else if (!h->cache->table[t->type].sets) { - h->cache->table[t->type].sets =nftnl_set_list_alloc (); - } -if (t && set) {struct nftnl_set *s =nftnl_set_alloc ();
Signature · 1 line
2.25.1
Re: [iptables PATCH v2 03/18] nft: cache: Init per table set list along with chain list
Quoted 2 lines · Phil Sutter
This simplifies code a bit and also aligns set and chain lists handling in cache.
Phil Sutter
reviewerPATCH v2 4/18 nft: cache: Fetch sets per table
In reply to Phil SutterKernel accepts a table name when dumping sets, so make use of that in case a table was passed to fetch_set_cache() but no set name.
--- iptables/nft-cache.c | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-)
iptables/nft-cache.c +15 -11
@@ -254,25 +254,31 @@ static int fetch_set_cache(struct nft_handle *h, .h = h, .t = t, }; +uint16_t flags =NLM_F_DUMP ; +struct nftnl_set *s =NULL ;struct nlmsghdr *nlh;char buf[16536 ];int i, ret; -if (t && set) { -struct nftnl_set *s =nftnl_set_alloc (); - +if (t) { + s =nftnl_set_alloc ();if (!s)return -1 ; - nlh =nftnl_set_nlmsg_build_hdr (buf,NFT_MSG_GETSET , h->family, -NLM_F_ACK , h->seq);nftnl_set_set_str (s,NFTNL_SET_TABLE , t->name); -nftnl_set_set_str (s,NFTNL_SET_NAME , set); + +if (set) { +nftnl_set_set_str (s,NFTNL_SET_NAME , set); + flags =NLM_F_ACK ; + } + } + + nlh =nftnl_set_nlmsg_build_hdr (buf,NFT_MSG_GETSET , + h->family, flags, h->seq); + +if (s) {nftnl_set_nlmsg_build_payload (nlh, s);nftnl_set_free (s); - }else { - nlh =nftnl_set_nlmsg_build_hdr (buf,NFT_MSG_GETSET , h->family, -NLM_F_DUMP , h->seq); } ret =mnl_talk (h, nlh, nftnl_set_list_cb, &d);
@@ -282,8 +288,6 @@ static int fetch_set_cache(struct nft_handle *h, }if (t && set) { -struct nftnl_set *s; - s =nftnl_set_list_lookup_byname (h->cache->table[t->type].sets, set);set_fetch_elem_cb (s, h);
Signature · 1 line
2.25.1
Re: [iptables PATCH v2 04/18] nft: cache: Fetch sets per table
Quoted 2 lines · Phil Sutter
Kernel accepts a table name when dumping sets, so make use of that in case a table was passed to
fetch_set_cache()but no set name.
Phil Sutter
reviewerPATCH v2 5/18 ebtables-restore: Table line to trigger implicit commit
In reply to Phil SutterCache code is suited for holding multiple tables' data at once. The only users of that are xtables-save and ebtables-restore with its support for multiple tables and lack of explicit COMMIT lines.
Remove the second user by introducing implicit commits upon table line parsing. This would allow to make cache single table only, but then xtables-save would fetch cache multiple times (once for each table) and therefore lose atomicity with regards to the acquired kernel ruleset image.
---
Changes since v1:
- Drop the custom table_new callback, committing from there is too late since table_flush happens before. Instead explicitly call commit() from parser.
--- iptables/xtables-restore.c | 4 ++++ 1 file changed, 4 insertions(+)
iptables/xtables-restore.c +4 -0
@@ -126,6 +126,10 @@ static void xtables_restore_parse_line(struct nft_handle *h,if (p->tablename && (strcmp (p->tablename, table) !=0 ))return ; +/* implicit commit if no explicit COMMIT supported */ +if (!p->commit) + cb->commit (h); +if (h->noflush ==0 ) {DEBUGP ("Cleaning all chains of table '%s'\n" , table);if (cb->table_flush)
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 6/18 nft: split parsing from netlink commands
In reply to Phil SutterFrom: Pablo Neira Ayuso <[email protected]>
This patch updates the parser to generate a list of command objects. This list of commands is then transformed to a list of netlink jobs. This new command object stores the rule using the nftnl representation via nft_rule_new().
To reduce the number of updates in this patch, the nft_*_rule_find() functions have been updated to restore the native representation to skip the update of the rule comparison code.
---
Changes since v1:
- Store rules' target chain name and check existence after fetching kernel ruleset to maintain compatibility with legacy iptables in error case. Consequently fold previous patch "nft: do not check for existing chain from parser" into this one.
- Drop unused struct iptables_command_state field from struct nft_cmd.
--- iptables/Makefile.am | 2 +- iptables/nft-arp.c | 5 +- iptables/nft-bridge.c | 5 +- iptables/nft-cmd.c | 319 ++++++++++++++++++ iptables/nft-cmd.h | 78 +++++ iptables/nft-shared.c | 6 +- iptables/nft-shared.h | 4 +- iptables/nft.c | 258 ++++++++++---- iptables/nft.h | 39 ++- .../testcases/ip6tables/0004-return-codes_0 | 1 + .../testcases/iptables/0004-return-codes_0 | 6 + iptables/xtables-arp.c | 26 +- iptables/xtables-eb.c | 26 +- iptables/xtables-restore.c | 26 +- iptables/xtables.c | 57 ++-- 15 files changed, 715 insertions(+), 143 deletions(-) create mode 100644 iptables/nft-cmd.c create mode 100644 iptables/nft-cmd.h
iptables/Makefile.am +1 -1
@@ -38,7 +38,7 @@ xtables_nft_multi_SOURCES += xtables-save.c xtables-restore.c \ nft-shared.c nft-ipv4.c nft-ipv6.c nft-arp.c \ xtables-monitor.c nft-cache.c \ xtables-arp-standalone.c xtables-arp.c \ - nft-bridge.c \ + nft-bridge.c nft-cmd.c \ xtables-eb-standalone.c xtables-eb.c \ xtables-eb-translate.c \ xtables-translate.c
iptables/nft-arp.c +3 -2
@@ -634,14 +634,15 @@ static bool nft_arp_is_same(const void *data_a, }static bool nft_arp_rule_find (struct nft_handle *h,struct nftnl_rule *r, -void *data) +struct nftnl_rule *rule) { -const struct iptables_command_state *cs = data; +struct iptables_command_state _cs = {}, *cs = &_cs;struct iptables_command_state this = {};bool ret = false;/* Delete by matching rule case */ nft_rule_to_iptables_command_state (h, r, &this); +nft_rule_to_iptables_command_state (h, rule, cs);if (!nft_arp_is_same (&cs->arp, &this.arp)) goto out;
iptables/nft-bridge.c +3 -2
@@ -748,13 +748,14 @@ static bool nft_bridge_is_same(const void *data_a, const void *data_b) }static bool nft_bridge_rule_find (struct nft_handle *h,struct nftnl_rule *r, -void *data) +struct nftnl_rule *rule) { -struct iptables_command_state *cs = data; +struct iptables_command_state _cs = {}, *cs = &_cs;struct iptables_command_state this = {};bool ret = false;nft_rule_to_ebtables_command_state (h, r, &this); +nft_rule_to_ebtables_command_state (h, rule, cs);DEBUGP ("comparing with... " );
iptables/nft-cmd.c new file +319 -0
319 lines not on this page · raw message
iptables/nft-cmd.h new file +78 -0
78 lines not on this page · raw message
iptables/nft-shared.c +4 -2
@@ -989,12 +989,14 @@ void nft_ipv46_parse_target(struct xtables_target *t, void *data) cs->target = t; } -bool nft_ipv46_rule_find (struct nft_handle *h,struct nftnl_rule *r,void *data) +bool nft_ipv46_rule_find (struct nft_handle *h,struct nftnl_rule *r, +struct nftnl_rule *rule) { -struct iptables_command_state *cs = data, this = {}; +struct iptables_command_state _cs = {}, this = {}, *cs = &_cs;bool ret = false;nft_rule_to_iptables_command_state (h, r, &this); +nft_rule_to_iptables_command_state (h, rule, cs);DEBUGP ("comparing with... " );#ifdef DEBUG_DEL
iptables/nft-shared.h +2 -2
@@ -110,7 +110,7 @@ struct nft_family_ops { struct iptables_command_state *cs);void (*clear_cs)(struct iptables_command_state *cs);bool (*rule_find)(struct nft_handle *h,struct nftnl_rule *r, -void *data); +struct nftnl_rule *rule);int (*xlate)(const void *data,struct xt_xlate *xl); };
@@ -172,7 +172,7 @@ struct nft_family_ops *nft_family_ops_lookup(int family);void nft_ipv46_parse_target (struct xtables_target *t,void *data);bool nft_ipv46_rule_find (struct nft_handle *h,struct nftnl_rule *r, -void *data); +struct nftnl_rule *rule);bool compare_matches (struct xtables_rule_match *mt1,struct xtables_rule_match *mt2);bool compare_targets (struct xtables_target *tg1,struct xtables_target *tg2);
iptables/nft.c +199 -59
471 lines not on this page · raw message
iptables/nft.h +34 -5
@@ -3,6 +3,8 @@#include "xshared.h" #include "nft-shared.h" +#include "nft-cache.h" +#include "nft-cmd.h" #include <libiptc/linux_list.h> enum nft_table_type {
@@ -45,6 +47,31 @@ struct nft_cache { } table[NFT_TABLE_MAX ]; }; +enum obj_update_type { +NFT_COMPAT_TABLE_ADD , +NFT_COMPAT_TABLE_FLUSH , +NFT_COMPAT_CHAIN_ADD , +NFT_COMPAT_CHAIN_USER_ADD , +NFT_COMPAT_CHAIN_USER_DEL , +NFT_COMPAT_CHAIN_USER_FLUSH , +NFT_COMPAT_CHAIN_UPDATE , +NFT_COMPAT_CHAIN_RENAME , +NFT_COMPAT_CHAIN_ZERO , +NFT_COMPAT_RULE_APPEND , +NFT_COMPAT_RULE_INSERT , +NFT_COMPAT_RULE_REPLACE , +NFT_COMPAT_RULE_DELETE , +NFT_COMPAT_RULE_FLUSH , +NFT_COMPAT_SET_ADD , +NFT_COMPAT_RULE_LIST , +NFT_COMPAT_RULE_CHECK , +NFT_COMPAT_CHAIN_RESTORE , +NFT_COMPAT_RULE_SAVE , +NFT_COMPAT_RULE_ZERO , +NFT_COMPAT_BRIDGE_USER_CHAIN_UPDATE , +NFT_COMPAT_TABLE_NEW , +}; +struct nft_handle {int family;struct mnl_socket *nl;
@@ -67,6 +94,7 @@ struct nft_handle {bool restore;bool noflush;int8_t config_done; +struct list_head cmd_list;/* meta data, for error reporting */ struct {
@@ -121,12 +149,13 @@ void nft_bridge_chain_postprocess(struct nft_handle *h,*/ struct nftnl_rule ; -int nft_rule_append (struct nft_handle *h,const char *chain,const char *table,void *data,struct nftnl_rule *ref,bool verbose); -int nft_rule_insert (struct nft_handle *h,const char *chain,const char *table,void *data,int rulenum,bool verbose); -int nft_rule_check (struct nft_handle *h,const char *chain,const char *table,void *data,bool verbose); -int nft_rule_delete (struct nft_handle *h,const char *chain,const char *table,void *data,bool verbose); +struct nftnl_rule *nft_rule_new (struct nft_handle *h,const char *chain,const char *table,void *data); +int nft_rule_append (struct nft_handle *h,const char *chain,const char *table,struct nftnl_rule *r,struct nftnl_rule *ref,bool verbose); +int nft_rule_insert (struct nft_handle *h,const char *chain,const char *table,struct nftnl_rule *r,int rulenum,bool verbose); +int nft_rule_check (struct nft_handle *h,const char *chain,const char *table,struct nftnl_rule *r,bool verbose); +int nft_rule_delete (struct nft_handle *h,const char *chain,const char *table,struct nftnl_rule *r,bool verbose);int nft_rule_delete_num (struct nft_handle *h,const char *chain,const char *table,int rulenum,bool verbose); -int nft_rule_replace (struct nft_handle *h,const char *chain,const char *table,void *data,int rulenum,bool verbose); +int nft_rule_replace (struct nft_handle *h,const char *chain,const char *table,struct nftnl_rule *r,int rulenum,bool verbose);int nft_rule_list (struct nft_handle *h,const char *chain,const char *table,int rulenum,unsigned int format);int nft_rule_list_save (struct nft_handle *h,const char *chain,const char *table,int rulenum,int counters);int nft_rule_save (struct nft_handle *h,const char *table,unsigned int format);
iptables/tests/shell/testcases/ip6tables/0004-return-codes_0 +1 -0
@@ -26,6 +26,7 @@ cmd 1 ip6tables -N foo # test rule adding cmd 0 ip6tables -A INPUT -j ACCEPT cmd 1 ip6tables -A noexist -j ACCEPT +cmd 2 ip6tables -I INPUT -j foobar # test rule checking cmd 0 ip6tables -C INPUT -j ACCEPT
iptables/tests/shell/testcases/iptables/0004-return-codes_0 +6 -0
@@ -54,10 +54,16 @@ cmd 1 "$ENOENT" iptables -Z bar # test chain rename cmd 0 iptables -E foo bar cmd 1 "$EEXIST_F" iptables -E foo bar +cmd 1 "$ENOENT" iptables -E foo bar2 +cmd 0 iptables -N foo2 +cmd 1 "$EEXIST_F" iptables -E foo2 bar # test rule adding cmd 0 iptables -A INPUT -j ACCEPT cmd 1 "$ENOENT" iptables -A noexist -j ACCEPT +cmd 2 "" iptables -I INPUT -j foobar +cmd 2 "" iptables -R INPUT 1 -j foobar +cmd 2 "" iptables -D INPUT -j foobar # test rulenum commands cmd 1 "$E2BIG_I" iptables -I INPUT 23 -j ACCEPT
iptables/xtables-arp.c +13 -13
@@ -400,7 +400,7 @@ list_entries(struct nft_handle *h, const char *chain, const char *table,if (linenumbers) format |=FMT_LINENUMBERS ; -return nft_rule_list (h, chain, table, rulenum, format); +return nft_cmd_rule_list (h, chain, table, rulenum, format); } static int
@@ -427,10 +427,10 @@ append_entry(struct nft_handle *h, cs->arp.arp.tgt.s_addr = daddrs[j].s_addr; cs->arp.arp.tmsk.s_addr = dmasks[j].s_addr;if (append) { - ret =nft_rule_append (h, chain, table, cs,NULL , + ret =nft_cmd_rule_append (h, chain, table, cs,NULL , verbose); }else { - ret =nft_rule_insert (h, chain, table, cs, + ret =nft_cmd_rule_insert (h, chain, table, cs, rulenum, verbose); } }
@@ -455,7 +455,7 @@ replace_entry(const char *chain, cs->arp.arp.smsk.s_addr = smask->s_addr; cs->arp.arp.tmsk.s_addr = dmask->s_addr; -return nft_rule_replace (h, chain, table, cs, rulenum, verbose); +return nft_cmd_rule_replace (h, chain, table, cs, rulenum, verbose); } static int
@@ -479,7 +479,7 @@ delete_entry(const char *chain,for (j =0 ; j < ndaddrs; j++) { cs->arp.arp.tgt.s_addr = daddrs[j].s_addr; cs->arp.arp.tmsk.s_addr = dmasks[j].s_addr; - ret =nft_rule_delete (h, chain, table, cs, verbose); + ret =nft_cmd_rule_delete (h, chain, table, cs, verbose); } }
@@ -955,7 +955,7 @@ int do_commandarp(struct nft_handle *h, int argc, char *argv[], char **table, options&OPT_VERBOSE, h);break ;case CMD_DELETE_NUM : - ret =nft_rule_delete_num (h, chain, *table, rulenum -1 , verbose); + ret =nft_cmd_rule_delete_num (h, chain, *table, rulenum -1 , verbose);break ;case CMD_REPLACE : ret = replace_entry(chain, *table, &cs, rulenum - 1,
@@ -977,10 +977,10 @@ int do_commandarp(struct nft_handle *h, int argc, char *argv[], char **table, options&OPT_LINENUMBERS);break ;case CMD_FLUSH : - ret =nft_rule_flush (h, chain, *table, options &OPT_VERBOSE ); + ret =nft_cmd_rule_flush (h, chain, *table, options &OPT_VERBOSE );break ;case CMD_ZERO : - ret =nft_chain_zero_counters (h, chain, *table, + ret =nft_cmd_chain_zero_counters (h, chain, *table, options &OPT_VERBOSE );break ;case CMD_LIST |CMD_ZERO :
@@ -990,21 +990,21 @@ int do_commandarp(struct nft_handle *h, int argc, char *argv[], char **table,/*options&OPT_EXPANDED*/ 0 , options&OPT_LINENUMBERS);if (ret) - ret =nft_chain_zero_counters (h, chain, *table, + ret =nft_cmd_chain_zero_counters (h, chain, *table, options &OPT_VERBOSE );break ;case CMD_NEW_CHAIN : - ret =nft_chain_user_add (h, chain, *table); + ret =nft_cmd_chain_user_add (h, chain, *table);break ;case CMD_DELETE_CHAIN : - ret =nft_chain_user_del (h, chain, *table, + ret =nft_cmd_chain_user_del (h, chain, *table, options &OPT_VERBOSE );break ;case CMD_RENAME_CHAIN : - ret =nft_chain_user_rename (h, chain, *table, newname); + ret =nft_cmd_chain_user_rename (h, chain, *table, newname);break ;case CMD_SET_POLICY : - ret =nft_chain_set (h, *table, chain, policy,NULL ); + ret =nft_cmd_chain_set (h, *table, chain, policy,NULL );if (ret <0 )xtables_error (PARAMETER_PROBLEM ,"Wrong policy `%s'\n" , policy);
iptables/xtables-eb.c +13 -13
@@ -150,9 +150,9 @@ append_entry(struct nft_handle *h,int ret =1 ;if (append) - ret =nft_rule_append (h, chain, table, cs,NULL , verbose); + ret =nft_cmd_rule_append (h, chain, table, cs,NULL , verbose);else - ret =nft_rule_insert (h, chain, table, cs, rule_nr, verbose); + ret =nft_cmd_rule_insert (h, chain, table, cs, rule_nr, verbose);return ret; }
@@ -169,10 +169,10 @@ delete_entry(struct nft_handle *h,int ret =1 ;if (rule_nr ==-1 ) - ret =nft_rule_delete (h, chain, table, cs, verbose); + ret =nft_cmd_rule_delete (h, chain, table, cs, verbose);else {do { - ret =nft_rule_delete_num (h, chain, table, + ret =nft_cmd_rule_delete_num (h, chain, table, rule_nr, verbose); rule_nr++; }while (rule_nr < rule_nr_end);
@@ -427,7 +427,7 @@ static int list_rules(struct nft_handle *h, const char *chain, const char *tableif (!counters) format |=FMT_NOCOUNTS ; -return nft_rule_list (h, chain, table, rule_nr, format); +return nft_cmd_rule_list (h, chain, table, rule_nr, format); }static int parse_rule_range (const char *argv,int *rule_nr,int *rule_nr_end)
@@ -813,7 +813,7 @@ int do_commandeb(struct nft_handle *h, int argc, char *argv[], char **table, flags |=OPT_COMMAND ;if (c =='N' ) { - ret =nft_chain_user_add (h, chain, *table); + ret =nft_cmd_chain_user_add (h, chain, *table);break ; }else if (c =='X' ) {/* X arg is optional, optarg is NULL */
@@ -821,7 +821,7 @@ int do_commandeb(struct nft_handle *h, int argc, char *argv[], char **table, chain = argv[optind]; optind++; } - ret =nft_chain_user_del (h, chain, *table,0 ); + ret =nft_cmd_chain_user_del (h, chain, *table,0 );break ; }
@@ -835,7 +835,7 @@ int do_commandeb(struct nft_handle *h, int argc, char *argv[], char **table,else if (strchr (argv[optind],' ' ) !=NULL )xtables_error (PARAMETER_PROBLEM ,"Use of ' ' not allowed in chain names" ); - ret =nft_chain_user_rename (h, chain, *table, + ret =nft_cmd_chain_user_rename (h, chain, *table, argv[optind]);if (ret !=0 && errno ==ENOENT )xtables_error (PARAMETER_PROBLEM ,"Chain '%s' doesn't exists" , chain);
@@ -1137,7 +1137,7 @@ print_zero:/*case 7 :*/ /* atomic-init */ /*case 10:*/ /* atomic-save */ case 11 :/* init-table */ -nft_table_flush (h, *table); +nft_cmd_table_flush (h, *table);return 1 ;/* replace->command = c;
@@ -1225,13 +1225,13 @@ print_zero:if (command =='P' ) {if (selected_chain >=NF_BR_NUMHOOKS ) { - ret =ebt_set_user_chain_policy (h, *table, chain, policy); + ret =ebt_cmd_user_chain_policy (h, *table, chain, policy); }else {if (strcmp (policy,"RETURN" ) ==0 ) {xtables_error (PARAMETER_PROBLEM ,"Policy RETURN only allowed for user defined chains" ); } - ret =nft_chain_set (h, *table, chain, policy,NULL ); + ret =nft_cmd_chain_set (h, *table, chain, policy,NULL );if (ret <0 )xtables_error (PARAMETER_PROBLEM ,"Wrong policy" ); }
@@ -1244,9 +1244,9 @@ print_zero: flags&LIST_C); }if (flags &OPT_ZERO ) { - ret =nft_chain_zero_counters (h, chain, *table,0 ); + ret =nft_cmd_chain_zero_counters (h, chain, *table,0 ); }else if (command =='F' ) { - ret =nft_rule_flush (h, chain, *table,0 ); + ret =nft_cmd_rule_flush (h, chain, *table,0 ); }else if (command =='A' ) { ret =append_entry (h, chain, *table, &cs,0 ,0 , true); }else if (command =='I' ) {
iptables/xtables-restore.c +13 -13
@@ -61,11 +61,11 @@ static void print_usage(const char *name, const char *version)static const struct nft_xt_restore_cb restore_cb = { .commit = nft_commit, .abort = nft_abort, - .table_new = nft_table_new, - .table_flush = nft_table_flush, + .table_new = nft_cmd_table_new, + .table_flush = nft_cmd_table_flush, .do_command = do_commandx, - .chain_set = nft_chain_set, - .chain_restore = nft_chain_restore, + .chain_set = nft_cmd_chain_set, + .chain_restore = nft_cmd_chain_restore, };struct nft_xt_restore_state {
@@ -193,7 +193,7 @@ static void xtables_restore_parse_line(struct nft_handle *h,"cannot create chain '%s' (%s)\n" , chain, strerror(errno)); }else if (h->family ==NFPROTO_BRIDGE && - !ebt_set_user_chain_policy (h, state->curtable->name, + !ebt_cmd_user_chain_policy (h, state->curtable->name, chain, policy)) { xtables_error(OTHER_PROBLEM, "Can't set policy `%s' on `%s' line %u: %s\n",
@@ -490,11 +490,11 @@ int xtables_ip6_restore_main(int argc, char *argv[])static const struct nft_xt_restore_cb ebt_restore_cb = { .commit = nft_bridge_commit, - .table_new = nft_table_new, - .table_flush = nft_table_flush, + .table_new = nft_cmd_table_new, + .table_flush = nft_cmd_table_flush, .do_command = do_commandeb, - .chain_set = nft_chain_set, - .chain_restore = nft_chain_restore, + .chain_set = nft_cmd_chain_set, + .chain_restore = nft_cmd_chain_restore, };static const struct option ebt_restore_options[] = {
@@ -536,11 +536,11 @@ int xtables_eb_restore_main(int argc, char *argv[])static const struct nft_xt_restore_cb arp_restore_cb = { .commit = nft_commit, - .table_new = nft_table_new, - .table_flush = nft_table_flush, + .table_new = nft_cmd_table_new, + .table_flush = nft_cmd_table_flush, .do_command = do_commandarp, - .chain_set = nft_chain_set, - .chain_restore = nft_chain_restore, + .chain_set = nft_cmd_chain_set, + .chain_restore = nft_cmd_chain_restore, };int xtables_arp_restore_main (int argc,char *argv[])
iptables/xtables.c +26 -31
169 lines not on this page · raw message
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 7/18 nft: calculate cache requirements from list of commands
In reply to Phil SutterFrom: Pablo Neira Ayuso <[email protected]>
This patch uses the new list of commands to calculate the cache requirements, the rationale after this updates is the following:
#1 Parsing, that builds the list of commands and it also calculates
cache level requirements.
#2 Cache building.
#3 Translate commands to jobs
#4 Translate jobs to netlink
This patch removes the pre-parsing code in xtables-restore.c to calculate the cache.
After this patch, cache is calculated only once, there is no need to cancel and refetch for an in-transit transaction.
---
Changes since v1:
- Rebased and resolved conflicts.
- Adjust for dropped init_chain_cache().
- Adjust for nft_rebuild_cache() respecting fake cache.
- Use if-clauses in __nft_build_cache() instead of switch() as that better fits the staged caching.
- Set cache_level to NFT_CL_TABLES in nft_cmd_table_flush(), not
NFT_CL_CHAINS: When flushing the whole table, chain list is not relevant.
- Move nft_cache_level_set() into nft-cache.c as it belongs there, make it public and have xtables_save_main() call it instead of manipulating h->cache_level directly.
- Actually drop input buffering in xtables-restore.c, it is not needed anymore.
--- iptables/nft-cache.c | 68 +++++++++++++---------------- iptables/nft-cache.h | 2 + iptables/nft-cmd.c | 66 +++++++++++++++++++++++++++++ iptables/nft-cmd.h | 1 + iptables/nft.c | 17 +++++++- iptables/nft.h | 2 +- iptables/xtables-restore.c | 87 +------------------------------------- iptables/xtables-save.c | 3 ++ 8 files changed, 119 insertions(+), 127 deletions(-)
iptables/nft-cache.c +29 -39
@@ -24,6 +24,14 @@#include "nft.h" #include "nft-cache.h" +void nft_cache_level_set (struct nft_handle *h,int level) +{ +if (level <= h->cache_level) +return ; + + h->cache_level = level; +} +static int genid_cb (const struct nlmsghdr *nlh,void *data) {uint32_t *genid = data;
@@ -436,42 +444,20 @@ __nft_build_cache(struct nft_handle *h, enum nft_cache_level level, const struct builtin_table *t, const char *set, const char *chain) { - if (level <= h->cache_level) + if (h->cache_init) return; -if (!h->nft_genid) -mnl_genid_get (h, &h->nft_genid); + h->cache_init = true; +mnl_genid_get (h, &h->nft_genid); -switch (h->cache_level) { -case NFT_CL_NONE : +if (h->cache_level >=NFT_CL_TABLES )fetch_table_cache (h); -if (level ==NFT_CL_TABLES ) -break ; -/* fall through */ -case NFT_CL_TABLES : +if (h->cache_level >=NFT_CL_CHAINS )fetch_chain_cache (h, t, chain); -if (level ==NFT_CL_CHAINS ) -break ; -/* fall through */ -case NFT_CL_CHAINS : +if (h->cache_level >=NFT_CL_SETS )fetch_set_cache (h, t, set); -if (level ==NFT_CL_SETS ) -break ; -/* fall through */ -case NFT_CL_SETS : +if (h->cache_level >=NFT_CL_RULES )fetch_rule_cache (h, t, chain); -if (level ==NFT_CL_RULES ) -break ; -/* fall through */ -case NFT_CL_RULES : -case NFT_CL_FAKE : -break ; - } - -if (!t && !chain) - h->cache_level = level; -else if (h->cache_level <NFT_CL_TABLES ) - h->cache_level =NFT_CL_TABLES ; }void nft_build_cache (struct nft_handle *h,struct nftnl_chain *c)
@@ -493,6 +479,7 @@ void nft_fake_cache(struct nft_handle *h)fetch_table_cache (h); h->cache_level =NFT_CL_FAKE ; + h->cache_init = true;mnl_genid_get (h, &h->nft_genid); }
@@ -593,26 +580,29 @@ static int flush_cache(struct nft_handle *h, struct nft_cache *c,void flush_chain_cache (struct nft_handle *h,const char *tablename) { -if (!h->cache_level) +if (!h->cache_init)return ;if (flush_cache (h, h->cache, tablename)) - h->cache_level =NFT_CL_NONE ; + h->cache_init = false; }void nft_rebuild_cache (struct nft_handle *h) { -enum nft_cache_level level = h->cache_level; - -if (h->cache_level) +if (h->cache_init) {__nft_flush_cache (h); + h->cache_init = false; + } -if (h->cache_level ==NFT_CL_FAKE ) { +if (h->cache_level ==NFT_CL_FAKE )nft_fake_cache (h); - }else { - h->cache_level =NFT_CL_NONE ; -__nft_build_cache (h, level,NULL ,NULL ,NULL ); - } +else +__nft_build_cache (h, h->cache_level,NULL ,NULL ,NULL ); +} + +void nft_cache_build (struct nft_handle *h) +{ +__nft_build_cache (h, h->cache_level,NULL ,NULL ,NULL ); }void nft_release_cache (struct nft_handle *h)
iptables/nft-cache.h +2 -0
@@ -3,6 +3,7 @@struct nft_handle ; +void nft_cache_level_set (struct nft_handle *h,int level);void nft_fake_cache (struct nft_handle *h);void nft_build_cache (struct nft_handle *h,struct nftnl_chain *c);void nft_rebuild_cache (struct nft_handle *h);
@@ -10,6 +11,7 @@ void nft_release_cache(struct nft_handle *h);void flush_chain_cache (struct nft_handle *h,const char *tablename);int flush_rule_cache (struct nft_handle *h,const char *table,struct nftnl_chain *c); +void nft_cache_build (struct nft_handle *h);struct nftnl_chain_list *nft_chain_list_get (struct nft_handle *h,const char *table,const char *chain);
iptables/nft-cmd.c +66 -0
185 lines not on this page · raw message
iptables/nft-cmd.h +1 -0
@@ -18,6 +18,7 @@ struct nft_cmd {unsigned int format;struct {struct nftnl_rule *rule; +struct nftnl_set *set; } obj;const char *policy;struct xt_counters counters;
iptables/nft.c +15 -2
@@ -954,6 +954,7 @@ static struct nftnl_set *add_anon_set(struct nft_handle *h, const char *table, {static uint32_t set_id =0 ;struct nftnl_set *s; +struct nft_cmd *cmd; s =nftnl_set_alloc ();if (!s)
@@ -969,7 +970,14 @@ static struct nftnl_set *add_anon_set(struct nft_handle *h, const char *table,nftnl_set_set_u32 (s,NFTNL_SET_KEY_LEN , key_len);nftnl_set_set_u32 (s,NFTNL_SET_DESC_SIZE , size); -return batch_set_add (h,NFT_COMPAT_SET_ADD , s) ? s :NULL ; + cmd =nft_cmd_new (h,NFT_COMPAT_SET_ADD , table,NULL ,NULL ,-1 , false); +if (!cmd) { +nftnl_set_free (s); +return NULL ; + } + cmd->obj.set = s; + +return s; } static struct nftnl_expr *
@@ -2996,6 +3004,8 @@ static int nft_prepare(struct nft_handle *h)struct nft_cmd *cmd, *next;int ret =1 ; +nft_cache_build (h); +list_for_each_entry_safe (cmd, next, &h->cmd_list, head) {switch (cmd->command) {case NFT_COMPAT_TABLE_FLUSH :
@@ -3081,9 +3091,12 @@ static int nft_prepare(struct nft_handle *h)nft_xt_builtin_init (h, cmd->table); ret =1 ;break ; +case NFT_COMPAT_SET_ADD : +batch_set_add (h,NFT_COMPAT_SET_ADD , cmd->obj.set); + ret =1 ; +break ;case NFT_COMPAT_TABLE_ADD :case NFT_COMPAT_CHAIN_ADD : -case NFT_COMPAT_SET_ADD :assert (0 );break ; }
iptables/nft.h +1 -1
@@ -30,7 +30,6 @@ struct builtin_table { };enum nft_cache_level { -NFT_CL_NONE ,NFT_CL_TABLES ,NFT_CL_CHAINS ,NFT_CL_SETS ,
@@ -95,6 +94,7 @@ struct nft_handle {bool noflush;int8_t config_done;struct list_head cmd_list; +bool cache_init;/* meta data, for error reporting */ struct {
iptables/xtables-restore.c +2 -85
101 lines not on this page · raw message
iptables/xtables-save.c +3 -0
@@ -239,6 +239,9 @@ xtables_save_main(int family, int argc, char *argv[],exit (EXIT_FAILURE ); } +nft_cache_level_set (&h,NFT_CL_RULES ); +nft_cache_build (&h); + ret =do_output (&h, tablename, &d);nft_fini (&h);if (dump)
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 8/18 nft: restore among support
In reply to Phil SutterFrom: Pablo Neira Ayuso <[email protected]>
Update among support to work again with the new parser and cache logic.
--- iptables/nft-bridge.c | 13 +++++++++++-- iptables/nft.c | 15 +++++++++++++++ iptables/nft.h | 6 ++++++ 3 files changed, 32 insertions(+), 2 deletions(-)
iptables/nft-bridge.c +11 -2
@@ -421,11 +421,20 @@ static struct nftnl_set *set_from_lookup_expr(struct nft_xt_ctx *ctx, const struct nftnl_expr *e) {const char *set_name =nftnl_expr_get_str (e,NFTNL_EXPR_LOOKUP_SET ); +uint32_t set_id =nftnl_expr_get_u32 (e,NFTNL_EXPR_LOOKUP_SET_ID );struct nftnl_set_list *slist; +struct nftnl_set *set; slist =nft_set_list_get (ctx->h, ctx->table, set_name); -if (slist) -return nftnl_set_list_lookup_byname (slist, set_name); +if (slist) { + set =nftnl_set_list_lookup_byname (slist, set_name); +if (set) +return set; + + set =nft_set_batch_lookup_byid (ctx->h, set_id); +if (set) +return set; + }return NULL ; }
iptables/nft.c +15 -0
@@ -1613,6 +1613,20 @@ int nft_rule_save(struct nft_handle *h, const char *table, unsigned int format)return ret ==0 ?1 :0 ; } +struct nftnl_set *nft_set_batch_lookup_byid (struct nft_handle *h, +uint32_t set_id) +{ +struct obj_update *n; + +list_for_each_entry (n, &h->obj_list, head) { +if (n->type ==NFT_COMPAT_SET_ADD && +nftnl_set_get_u32 (n->set,NFTNL_SET_ID ) == set_id) +return n->set; + } + +return NULL ; +} +static void __nft_rule_flush (struct nft_handle *h,const char *table,const char *chain,bool verbose,bool implicit)
@@ -3092,6 +3106,7 @@ static int nft_prepare(struct nft_handle *h) ret =1 ;break ;case NFT_COMPAT_SET_ADD : +nft_xt_builtin_init (h, cmd->table);batch_set_add (h,NFT_COMPAT_SET_ADD , cmd->obj.set); ret =1 ;break ;
iptables/nft.h +6 -0
@@ -144,6 +144,12 @@ void nft_bridge_chain_postprocess(struct nft_handle *h, struct nftnl_chain *c); +/* +* Operations with sets. +*/ +struct nftnl_set *nft_set_batch_lookup_byid (struct nft_handle *h, +uint32_t set_id); +/* * Operations with rule-set. */
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 9/18 nft: remove cache build calls
In reply to Phil SutterFrom: Pablo Neira Ayuso <[email protected]>
The cache requirements are now calculated once from the parsing phase. There is no need to call __nft_build_cache() from several spots in the codepath anymore.
---
Changes since v1:
- Drop now unused nft_build_cache() function.
--- iptables/nft-cache.c | 20 -------------------- iptables/nft-cache.h | 1 - iptables/nft.c | 21 --------------------- 3 files changed, 42 deletions(-)
iptables/nft-cache.c +0 -20
@@ -460,20 +460,6 @@ __nft_build_cache(struct nft_handle *h, enum nft_cache_level level,fetch_rule_cache (h, t, chain); } -void nft_build_cache (struct nft_handle *h,struct nftnl_chain *c) -{ -const struct builtin_table *t; -const char *table, *chain; - -if (!c) -return __nft_build_cache (h,NFT_CL_RULES ,NULL ,NULL ,NULL ); - - table =nftnl_chain_get_str (c,NFTNL_CHAIN_TABLE ); - chain =nftnl_chain_get_str (c,NFTNL_CHAIN_NAME ); - t =nft_table_builtin_find (h, table); -__nft_build_cache (h,NFT_CL_RULES , t,NULL , chain); -} -void nft_fake_cache (struct nft_handle *h) {fetch_table_cache (h);
@@ -619,8 +605,6 @@ void nft_release_cache(struct nft_handle *h)struct nftnl_table_list *nftnl_table_list_get (struct nft_handle *h) { -__nft_build_cache (h,NFT_CL_TABLES ,NULL ,NULL ,NULL ); -return h->cache->tables; }
@@ -633,8 +617,6 @@ nft_set_list_get(struct nft_handle *h, const char *table, const char *set)if (!t)return NULL ; -__nft_build_cache (h,NFT_CL_RULES , t, set,NULL ); -return h->cache->table[t->type].sets; }
@@ -647,8 +629,6 @@ nft_chain_list_get(struct nft_handle *h, const char *table, const char *chain)if (!t)return NULL ; -__nft_build_cache (h,NFT_CL_CHAINS , t,NULL , chain); -return h->cache->table[t->type].chains; }
iptables/nft-cache.h +0 -1
@@ -5,7 +5,6 @@ struct nft_handle;void nft_cache_level_set (struct nft_handle *h,int level);void nft_fake_cache (struct nft_handle *h); -void nft_build_cache (struct nft_handle *h,struct nftnl_chain *c);void nft_rebuild_cache (struct nft_handle *h);void nft_release_cache (struct nft_handle *h);void flush_chain_cache (struct nft_handle *h,const char *tablename);
iptables/nft.c +0 -21
@@ -1367,14 +1367,6 @@ nft_rule_append(struct nft_handle *h, const char *chain, const char *table,nft_xt_builtin_init (h, table); -/* Since ebtables user-defined chain policies are implemented as last -* rule in nftables, rule cache is required here to treat them right. */ -if (h->family ==NFPROTO_BRIDGE ) { - c =nft_chain_find (h, table, chain); -if (c && !nft_chain_builtin (c)) -nft_build_cache (h, c); - } - nft_fn = nft_rule_append;if (ref) {
@@ -1599,7 +1591,6 @@ int nft_rule_save(struct nft_handle *h, const char *table, unsigned int format) c =nftnl_chain_list_iter_next (iter);while (c) { -nft_build_cache (h, c); ret =nft_chain_save_rules (h, c, format);if (ret !=0 )break ;
@@ -1807,10 +1798,6 @@ static int __nft_chain_user_del(struct nftnl_chain *c, void *data)fprintf (stdout,"Deleting chain `%s'\n" ,nftnl_chain_get_str (c,NFTNL_CHAIN_NAME )); -/* This triggers required policy rule deletion. */ -if (h->family ==NFPROTO_BRIDGE ) -nft_build_cache (h, c); -/* XXX This triggers a fast lookup from the kernel. */ nftnl_chain_unset (c,NFTNL_CHAIN_HANDLE ); ret =batch_chain_add (h,NFT_COMPAT_CHAIN_USER_DEL , c);
@@ -2093,8 +2080,6 @@ nft_rule_find(struct nft_handle *h, struct nftnl_chain *c,struct nftnl_rule_iter *iter;bool found = false; -nft_build_cache (h, c); -if (rulenum >=0 )/* Delete by rule number case */ return nftnl_rule_lookup_byindex (c, rulenum);
@@ -2979,8 +2964,6 @@ int ebt_set_user_chain_policy(struct nft_handle *h, const char *table,else return 0 ; -nft_build_cache (h, c); -nftnl_chain_set_u32 (c,NFTNL_CHAIN_POLICY , pval);return 1 ; }
@@ -3333,8 +3316,6 @@ static int __nft_chain_zero_counters(struct nftnl_chain *c, void *data)return -1 ; } -nft_build_cache (h, c); - iter =nftnl_rule_iter_create (c);if (iter ==NULL )return -1 ;
@@ -3471,8 +3452,6 @@ static int nft_is_chain_compatible(struct nftnl_chain *c, void *data)enum nf_inet_hooks hook;int prio; -nft_build_cache (h, c); -if (nftnl_rule_foreach (c, nft_is_rule_compatible,NULL ))return -1 ;
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 10/18 nft: missing nft_fini() call in bridge family
In reply to Phil SutterFrom: Pablo Neira Ayuso <[email protected]>
--- iptables/xtables-eb-standalone.c | 2 ++ 1 file changed, 2 insertions(+)
iptables/xtables-eb-standalone.c +2 -0
@@ -53,6 +53,8 @@ int xtables_eb_main(int argc, char *argv[])if (ret) ret =nft_bridge_commit (&h); +nft_fini (&h); +if (!ret)fprintf (stderr,"ebtables: %s\n" ,nft_strerror (errno));
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 11/18 nft: cache: Simplify rule and set fetchers
In reply to Phil SutterSince no incremental cache fetching happens anymore, code fetching rules for chains or elements for sets may safely assume that whatever is in cache also didn't get populated with rules or elements before.
Therefore no (optional) chain name needs to be passed on to fetch_rule_cache() and fetch_set_cache() doesn't have to select for which sets in a table to call set_fetch_elem_cb().
--- iptables/nft-cache.c | 20 +++++--------------- 1 file changed, 5 insertions(+), 15 deletions(-)
iptables/nft-cache.c +5 -15
@@ -295,11 +295,7 @@ static int fetch_set_cache(struct nft_handle *h,return ret; } -if (t && set) { - s =nftnl_set_list_lookup_byname (h->cache->table[t->type].sets, - set); -set_fetch_elem_cb (s, h); - }else if (t) { +if (t) {nftnl_set_list_foreach (h->cache->table[t->type].sets, set_fetch_elem_cb, h); }else {
@@ -409,20 +405,14 @@ static int nft_rule_list_update(struct nftnl_chain *c, void *data) }static int fetch_rule_cache (struct nft_handle *h, -const struct builtin_table *t,const char *chain) +const struct builtin_table *t) {int i;if (t) { -struct nftnl_chain_list *list; -struct nftnl_chain *c; - - list = h->cache->table[t->type].chains; +struct nftnl_chain_list *list = + h->cache->table[t->type].chains; -if (chain) { - c =nftnl_chain_list_lookup_byname (list, chain); -return nft_rule_list_update (c, h); - }return nftnl_chain_list_foreach (list, nft_rule_list_update, h); }
@@ -457,7 +447,7 @@ __nft_build_cache(struct nft_handle *h, enum nft_cache_level level,if (h->cache_level >=NFT_CL_SETS )fetch_set_cache (h, t, set);if (h->cache_level >=NFT_CL_RULES ) -fetch_rule_cache (h, t, chain); +fetch_rule_cache (h, t); }void nft_fake_cache (struct nft_handle *h)
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 12/18 nft: cache: Improve fake cache integration
In reply to Phil SutterWith NFT_CL_FAKE being highest cache level while at the same time __nft_build_cache() treating it equal to NFT_CL_TABLES, no special handling for fake cache is required anymore.
--- iptables/nft-cache.c | 16 +++------------- iptables/nft-cache.h | 1 - iptables/xtables-restore.c | 2 +- 3 files changed, 4 insertions(+), 15 deletions(-)
iptables/nft-cache.c +3 -13
@@ -442,6 +442,8 @@ __nft_build_cache(struct nft_handle *h, enum nft_cache_level level,if (h->cache_level >=NFT_CL_TABLES )fetch_table_cache (h); +if (h->cache_level ==NFT_CL_FAKE ) +return ;if (h->cache_level >=NFT_CL_CHAINS )fetch_chain_cache (h, t, chain);if (h->cache_level >=NFT_CL_SETS )
@@ -450,15 +452,6 @@ __nft_build_cache(struct nft_handle *h, enum nft_cache_level level,fetch_rule_cache (h, t); } -void nft_fake_cache (struct nft_handle *h) -{ -fetch_table_cache (h); - - h->cache_level =NFT_CL_FAKE ; - h->cache_init = true; -mnl_genid_get (h, &h->nft_genid); -} -static void __nft_flush_cache (struct nft_handle *h) {if (!h->cache_index) {
@@ -570,10 +563,7 @@ void nft_rebuild_cache(struct nft_handle *h) h->cache_init = false; } -if (h->cache_level ==NFT_CL_FAKE ) -nft_fake_cache (h); -else -__nft_build_cache (h, h->cache_level,NULL ,NULL ,NULL ); +__nft_build_cache (h, h->cache_level,NULL ,NULL ,NULL ); }void nft_cache_build (struct nft_handle *h)
iptables/nft-cache.h +0 -1
@@ -4,7 +4,6 @@struct nft_handle ;void nft_cache_level_set (struct nft_handle *h,int level); -void nft_fake_cache (struct nft_handle *h);void nft_rebuild_cache (struct nft_handle *h);void nft_release_cache (struct nft_handle *h);void flush_chain_cache (struct nft_handle *h,const char *tablename);
iptables/xtables-restore.c +1 -1
@@ -259,7 +259,7 @@ void xtables_restore_parse(struct nft_handle *h,char buffer[10240 ] = {};if (!h->noflush) -nft_fake_cache (h); +nft_cache_level_set (h,NFT_CL_FAKE ); line =0 ;while (fgets (buffer,sizeof (buffer), p->in)) {
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 13/18 nft: cache: Introduce struct nft_cache_req
In reply to Phil SutterThis embedded struct collects cache requirement info gathered from parsed nft_cmds and is interpreted by __nft_build_cache().
While being at it, remove unused parameters passed to the latter function, nft_handle pointer is sufficient.
--- iptables/nft-cache.c | 32 +++++++++++++++++--------------- iptables/nft.h | 6 +++++- 2 files changed, 22 insertions(+), 16 deletions(-)
iptables/nft-cache.c +17 -15
@@ -26,10 +26,12 @@void nft_cache_level_set (struct nft_handle *h,int level) { -if (level <= h->cache_level) +struct nft_cache_req *req = &h->cache_req; + +if (level <= req->level)return ; - h->cache_level = level; + req->level = level; }static int genid_cb (const struct nlmsghdr *nlh,void *data)
@@ -430,26 +432,26 @@ static int fetch_rule_cache(struct nft_handle *h, }static void -__nft_build_cache (struct nft_handle *h,enum nft_cache_level level, -const struct builtin_table *t,const char *set, -const char *chain) +__nft_build_cache (struct nft_handle *h) { +struct nft_cache_req *req = &h->cache_req; +if (h->cache_init)return ; h->cache_init = true;mnl_genid_get (h, &h->nft_genid); -if (h->cache_level >=NFT_CL_TABLES ) +if (req->level >=NFT_CL_TABLES )fetch_table_cache (h); -if (h->cache_level ==NFT_CL_FAKE ) +if (req->level ==NFT_CL_FAKE )return ; -if (h->cache_level >=NFT_CL_CHAINS ) -fetch_chain_cache (h, t, chain); -if (h->cache_level >=NFT_CL_SETS ) -fetch_set_cache (h, t, set); -if (h->cache_level >=NFT_CL_RULES ) -fetch_rule_cache (h, t); +if (req->level >=NFT_CL_CHAINS ) +fetch_chain_cache (h,NULL ,NULL ); +if (req->level >=NFT_CL_SETS ) +fetch_set_cache (h,NULL ,NULL ); +if (req->level >=NFT_CL_RULES ) +fetch_rule_cache (h,NULL ); }static void __nft_flush_cache (struct nft_handle *h)
@@ -563,12 +565,12 @@ void nft_rebuild_cache(struct nft_handle *h) h->cache_init = false; } -__nft_build_cache (h, h->cache_level,NULL ,NULL ,NULL ); +__nft_build_cache (h); }void nft_cache_build (struct nft_handle *h) { -__nft_build_cache (h, h->cache_level,NULL ,NULL ,NULL ); +__nft_build_cache (h); }void nft_release_cache (struct nft_handle *h)
iptables/nft.h +5 -1
@@ -71,6 +71,10 @@ enum obj_update_type { NFT_COMPAT_TABLE_NEW, }; +struct nft_cache_req { +enum nft_cache_level level; +}; +struct nft_handle {int family;struct mnl_socket *nl;
@@ -89,7 +93,7 @@ struct nft_handle {unsigned int cache_index;struct nft_cache __cache[2 ];struct nft_cache *cache; -enum nft_cache_level cache_level; +struct nft_cache_req cache_req;bool restore;bool noflush;int8_t config_done;
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 14/18 nft-cache: Fetch cache per table
In reply to Phil SutterRestore per-table operation of cache routines as initially implemented in commit e2883c5531e6e ("nft-cache: Support partial cache per table").
As before, this doesn't limit fetching of tables (their number is supposed to be low) but instead limits fetching of sets, chains and rules to the specified table.
For this to behave correctly when restoring without flushing over multiple tables, cache must be freed fully after each commit - otherwise the previous table's cache level is reused for the current one. The exception being fake cache, used for flushing restore: NFT_CL_FAKE is set just once at program startup, so it must stay set otherwise consecutive tables cause pointless cache fetching.
The sole use-case requiring a multi-table cache, iptables-save, is indicated by req->table being NULL. Therefore, req->table assignment is a bit sloppy: All calls to nft_cache_level_set() are assumed to set the same table value, no collision detection happens.
--- iptables/nft-cache.c | 33 ++++++++++++++-------- iptables/nft-cache.h | 4 ++- iptables/nft-cmd.c | 57 +++++++++++++++++++------------------- iptables/nft.h | 1 + iptables/xtables-restore.c | 2 +- iptables/xtables-save.c | 2 +- 6 files changed, 56 insertions(+), 43 deletions(-)
iptables/nft-cache.c +22 -11
@@ -24,14 +24,18 @@#include "nft.h" #include "nft-cache.h" -void nft_cache_level_set (struct nft_handle *h,int level) +void nft_cache_level_set (struct nft_handle *h,int level, +const struct nft_cmd *cmd) {struct nft_cache_req *req = &h->cache_req; -if (level <= req->level) +if (level > req->level) + req->level = level; + +if (!cmd)return ; - req->level = level; + req->table = cmd->table; }static int genid_cb (const struct nlmsghdr *nlh,void *data)
@@ -435,10 +439,14 @@ static void __nft_build_cache(struct nft_handle *h) {struct nft_cache_req *req = &h->cache_req; +const struct builtin_table *t =NULL ;if (h->cache_init)return ; +if (req->table) + t =nft_table_builtin_find (h, req->table); + h->cache_init = true;mnl_genid_get (h, &h->nft_genid);
@@ -447,11 +455,11 @@ __nft_build_cache(struct nft_handle *h)if (req->level ==NFT_CL_FAKE )return ;if (req->level >=NFT_CL_CHAINS ) -fetch_chain_cache (h,NULL ,NULL ); +fetch_chain_cache (h, t,NULL );if (req->level >=NFT_CL_SETS ) -fetch_set_cache (h,NULL ,NULL ); +fetch_set_cache (h, t,NULL );if (req->level >=NFT_CL_RULES ) -fetch_rule_cache (h,NULL ); +fetch_rule_cache (h, t); }static void __nft_flush_cache (struct nft_handle *h)
@@ -575,14 +583,17 @@ void nft_cache_build(struct nft_handle *h)void nft_release_cache (struct nft_handle *h) { -if (!h->cache_index) -return ; +struct nft_cache_req *req = &h->cache_req; +while (h->cache_index) +flush_cache (h, &h->__cache[h->cache_index--],NULL );flush_cache (h, &h->__cache[0 ],NULL ); -memcpy (&h->__cache[0 ], &h->__cache[1 ],sizeof (h->__cache[0 ])); -memset (&h->__cache[1 ],0 ,sizeof (h->__cache[1 ])); - h->cache_index =0 ; h->cache = &h->__cache[0 ]; + h->cache_init = false; + +if (req->level !=NFT_CL_FAKE ) + req->level =NFT_CL_TABLES ; + req->table =NULL ; }struct nftnl_table_list *nftnl_table_list_get (struct nft_handle *h)
iptables/nft-cache.h +3 -1
@@ -2,8 +2,10 @@#define _NFT_CACHE_H_struct nft_handle ; +struct nft_cmd ; -void nft_cache_level_set (struct nft_handle *h,int level); +void nft_cache_level_set (struct nft_handle *h,int level, +const struct nft_cmd *cmd);void nft_rebuild_cache (struct nft_handle *h);void nft_release_cache (struct nft_handle *h);void flush_chain_cache (struct nft_handle *h,const char *tablename);
iptables/nft-cmd.c +28 -29
194 lines not on this page · raw message
iptables/nft.h +1 -0
@@ -73,6 +73,7 @@ enum obj_update_type {struct nft_cache_req {enum nft_cache_level level; +const char *table; };struct nft_handle {
iptables/xtables-restore.c +1 -1
@@ -259,7 +259,7 @@ void xtables_restore_parse(struct nft_handle *h,char buffer[10240 ] = {};if (!h->noflush) -nft_cache_level_set (h,NFT_CL_FAKE ); +nft_cache_level_set (h,NFT_CL_FAKE ,NULL ); line =0 ;while (fgets (buffer,sizeof (buffer), p->in)) {
iptables/xtables-save.c +1 -1
@@ -239,7 +239,7 @@ xtables_save_main(int family, int argc, char *argv[],exit (EXIT_FAILURE ); } -nft_cache_level_set (&h,NFT_CL_RULES ); +nft_cache_level_set (&h,NFT_CL_RULES ,NULL );nft_cache_build (&h); ret =do_output (&h, tablename, &d);
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 15/18 nft-cache: Introduce __fetch_chain_cache()
In reply to Phil SutterExtract the inner part of fetch_chain_cache() into a dedicated function, preparing for individual chain caching.
--- iptables/nft-cache.c | 50 ++++++++++++++++++++++++++------------------ 1 file changed, 30 insertions(+), 20 deletions(-)
iptables/nft-cache.c +30 -20
@@ -318,9 +318,9 @@ static int fetch_set_cache(struct nft_handle *h,return ret; } -static int fetch_chain_cache (struct nft_handle *h, -const struct builtin_table *t, -const char *chain) +static int __fetch_chain_cache (struct nft_handle *h, +const struct builtin_table *t, +const struct nftnl_chain *c) {struct nftnl_chain_list_cb_data d = { .h = h,
@@ -330,24 +330,10 @@ static int fetch_chain_cache(struct nft_handle *h,struct nlmsghdr *nlh;int ret; -if (t && chain) { -struct nftnl_chain *c =nftnl_chain_alloc (); - -if (!c) -return -1 ; - - nlh =nftnl_chain_nlmsg_build_hdr (buf,NFT_MSG_GETCHAIN , - h->family,NLM_F_ACK , - h->seq); -nftnl_chain_set_str (c,NFTNL_CHAIN_TABLE , t->name); -nftnl_chain_set_str (c,NFTNL_CHAIN_NAME , chain); + nlh =nftnl_chain_nlmsg_build_hdr (buf,NFT_MSG_GETCHAIN , h->family, + c ?NLM_F_ACK :NLM_F_DUMP , h->seq); +if (c)nftnl_chain_nlmsg_build_payload (nlh, c); -nftnl_chain_free (c); - }else { - nlh =nftnl_chain_nlmsg_build_hdr (buf,NFT_MSG_GETCHAIN , - h->family,NLM_F_DUMP , - h->seq); - } ret =mnl_talk (h, nlh, nftnl_chain_list_cb, &d);if (ret <0 && errno ==EINTR )
@@ -356,6 +342,30 @@ static int fetch_chain_cache(struct nft_handle *h,return ret; } +static int fetch_chain_cache (struct nft_handle *h, +const struct builtin_table *t, +const char *chain) +{ +struct nftnl_chain *c; +int ret; + +if (!chain) +return __fetch_chain_cache (h, t,NULL ); + +assert (t); + + c =nftnl_chain_alloc (); +if (!c) +return -1 ; + +nftnl_chain_set_str (c,NFTNL_CHAIN_TABLE , t->name); +nftnl_chain_set_str (c,NFTNL_CHAIN_NAME , chain); + ret =__fetch_chain_cache (h, t, c); + +nftnl_chain_free (c); +return ret; +} +static int nftnl_rule_list_cb (const struct nlmsghdr *nlh,void *data) {struct nftnl_chain *c = data;
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 16/18 nft: cache: Fetch cache for specific chains
In reply to Phil SutterIterate over command list and collect chains to cache. Insert them into a sorted list to pass to __nft_build_cache().
If a command is interested in all chains (e.g., --list), cmd->chain remains unset. To record this case reliably, use a boolean ('all_chains'). Otherwise, it is hard to distinguish between first call to nft_cache_level_set() and previous command with NULL cmd->chain value.
When caching only specific chains, manually add builtin ones for the given table as well - otherwise nft_xt_builtin_init() will act as if they don't exist and possibly override non-default chain policies.
--- iptables/nft-cache.c | 82 +++++++++++++++++++++++++++++++++++++++----- iptables/nft.c | 1 + iptables/nft.h | 7 ++++ 3 files changed, 82 insertions(+), 8 deletions(-)
iptables/nft-cache.c +74 -8
@@ -11,6 +11,7 @@#include <assert.h> #include <errno.h> +#include <stdlib.h> #include <string.h> #include <xtables.h>
@@ -24,6 +25,24 @@#include "nft.h" #include "nft-cache.h" +static void cache_chain_list_insert (struct list_head *list,const char *name) +{ +struct cache_chain *pos =NULL , *new; + +list_for_each_entry (pos, list, head) { +int cmp =strcmp (pos->name, name); + +if (!cmp) +return ; +if (cmp >0 ) +break ; + } + + new =xtables_malloc (sizeof (*new)); + new->name = name; +list_add_tail (&new->head, pos ? &pos->head : list); +} +void nft_cache_level_set (struct nft_handle *h,int level,const struct nft_cmd *cmd) {
@@ -32,10 +51,21 @@ void nft_cache_level_set(struct nft_handle *h, int level,if (level > req->level) req->level = level; -if (!cmd) +if (!cmd || !cmd->table || req->all_chains)return ; req->table = cmd->table; + +if (!cmd->chain) { + req->all_chains = true; +return ; + } + +cache_chain_list_insert (&req->chain_list, cmd->chain); +if (cmd->rename) +cache_chain_list_insert (&req->chain_list, cmd->rename); +if (cmd->jumpto) +cache_chain_list_insert (&req->chain_list, cmd->jumpto); }static int genid_cb (const struct nlmsghdr *nlh,void *data)
@@ -344,12 +374,13 @@ static int __fetch_chain_cache(struct nft_handle *h,static int fetch_chain_cache (struct nft_handle *h,const struct builtin_table *t, -const char *chain) +struct list_head *chains) { +struct cache_chain *cc;struct nftnl_chain *c; -int ret; +int rc, ret =0 ; -if (!chain) +if (!chains)return __fetch_chain_cache (h, t,NULL );assert (t);
@@ -359,8 +390,13 @@ static int fetch_chain_cache(struct nft_handle *h,return -1 ;nftnl_chain_set_str (c,NFTNL_CHAIN_TABLE , t->name); -nftnl_chain_set_str (c,NFTNL_CHAIN_NAME , chain); - ret =__fetch_chain_cache (h, t, c); + +list_for_each_entry (cc, chains, head) { +nftnl_chain_set_str (c,NFTNL_CHAIN_NAME , cc->name); + rc =__fetch_chain_cache (h, t, c); +if (rc) + ret = rc; + }nftnl_chain_free (c);return ret;
@@ -450,12 +486,16 @@ __nft_build_cache(struct nft_handle *h) {struct nft_cache_req *req = &h->cache_req;const struct builtin_table *t =NULL ; +struct list_head *chains =NULL ;if (h->cache_init)return ; -if (req->table) +if (req->table) { t =nft_table_builtin_find (h, req->table); +if (!req->all_chains) + chains = &req->chain_list; + } h->cache_init = true;mnl_genid_get (h, &h->nft_genid);
@@ -465,7 +505,7 @@ __nft_build_cache(struct nft_handle *h)if (req->level ==NFT_CL_FAKE )return ;if (req->level >=NFT_CL_CHAINS ) -fetch_chain_cache (h, t,NULL ); +fetch_chain_cache (h, t, chains);if (req->level >=NFT_CL_SETS )fetch_set_cache (h, t,NULL );if (req->level >=NFT_CL_RULES )
@@ -588,12 +628,32 @@ void nft_rebuild_cache(struct nft_handle *h)void nft_cache_build (struct nft_handle *h) { +struct nft_cache_req *req = &h->cache_req; +const struct builtin_table *t =NULL ; +int i; + +if (req->table) + t =nft_table_builtin_find (h, req->table); + +/* fetch builtin chains as well (if existing) so nft_xt_builtin_init() +* doesn't override policies by accident */ +if (t && !req->all_chains) { +for (i =0 ; i <NF_INET_NUMHOOKS ; i++) { +const char *cname = t->chains[i].name; + +if (!cname) +break ; +cache_chain_list_insert (&req->chain_list, cname); + } + } +__nft_build_cache (h); }void nft_release_cache (struct nft_handle *h) {struct nft_cache_req *req = &h->cache_req; +struct cache_chain *cc, *cc_tmp;while (h->cache_index)flush_cache (h, &h->__cache[h->cache_index--],NULL );
@@ -604,6 +664,12 @@ void nft_release_cache(struct nft_handle *h)if (req->level !=NFT_CL_FAKE ) req->level =NFT_CL_TABLES ; req->table =NULL ; + + req->all_chains = false; +list_for_each_entry_safe (cc, cc_tmp, &req->chain_list, head) { +list_del (&cc->head); +free (cc); + } }struct nftnl_table_list *nftnl_table_list_get (struct nft_handle *h)
iptables/nft.c +1 -0
@@ -805,6 +805,7 @@ int nft_init(struct nft_handle *h, int family, const struct builtin_table *t)INIT_LIST_HEAD (&h->obj_list);INIT_LIST_HEAD (&h->err_list);INIT_LIST_HEAD (&h->cmd_list); +INIT_LIST_HEAD (&h->cache_req.chain_list);return 0 ; }
iptables/nft.h +7 -0
@@ -71,9 +71,16 @@ enum obj_update_type { NFT_COMPAT_TABLE_NEW, }; +struct cache_chain { +struct list_head head; +const char *name; +}; +struct nft_cache_req {enum nft_cache_level level;const char *table; +bool all_chains; +struct list_head chain_list; };struct nft_handle {
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 17/18 nft: cache: Optimize caching for flush command
In reply to Phil SutterWhen flushing all chains and verbose mode is not enabled, nft_rule_flush() uses a shortcut: It doesn't specify a chain name for NFT_MSG_DELRULE, so the kernel will flush all existing chains without user space needing to know which they are.
The above allows to avoid a chain cache, but there's a caveat:nft_xt_builtin_init() will create base chains as it assumes they are missing and thereby possibly overrides any non-default chain policies.
Solve this by making nft_xt_builtin_init() cache-aware: If a command doesn't need a chain cache, there's no need to bother with creating any non-existing builtin chains, either. For the sake of completeness, also do nothing if cache is not initialized (although that shouldn't happen).
--- iptables/nft-cmd.c | 5 +++- iptables/nft.c | 6 ++++ .../testcases/nft-only/0006-policy-override_0 | 29 +++++++++++++++++++ 3 files changed, 39 insertions(+), 1 deletion(-) create mode 100755 iptables/tests/shell/testcases/nft-only/0006-policy-override_0
iptables/nft-cmd.c +4 -1
@@ -159,7 +159,10 @@ int nft_cmd_rule_flush(struct nft_handle *h, const char *chain,if (!cmd)return 0 ; -nft_cache_level_set (h,NFT_CL_CHAINS , cmd); +if (chain || verbose) +nft_cache_level_set (h,NFT_CL_CHAINS , cmd); +else +nft_cache_level_set (h,NFT_CL_TABLES , cmd);return 1 ; }
iptables/nft.c +6 -0
@@ -737,6 +737,9 @@ static int nft_xt_builtin_init(struct nft_handle *h, const char *table) {const struct builtin_table *t; +if (!h->cache_init) +return 0 ; + t =nft_table_builtin_find (h, table);if (t ==NULL )return -1 ;
@@ -747,6 +750,9 @@ static int nft_xt_builtin_init(struct nft_handle *h, const char *table)if (nft_table_builtin_add (h, t) <0 )return -1 ; +if (h->cache_req.level <NFT_CL_CHAINS ) +return 0 ; +nft_chain_builtin_init (h, t); h->cache->table[t->type].initialized = true;
iptables/tests/shell/testcases/nft-only/0006-policy-override_0 new file +29 -0
@@ -0,0 +1,29 @@ +#!/bin/bash + +[[ $XT_MULTI == *xtables-nft-multi ]] || { echo "skip $XT_MULTI"; exit 0; } + +# make sure none of the commands invoking nft_xt_builtin_init() override +# non-default chain policies via needless chain add. + +RC=0 + +do_test() { + $XT_MULTI $@ + $XT_MULTI iptables -S | grep -q -- '-P FORWARD DROP' && return + + echo "command '$@' kills chain policies" + $XT_MULTI iptables -P FORWARD DROP + RC=1 +} + +$XT_MULTI iptables -P FORWARD DROP + +do_test iptables -A OUTPUT -j ACCEPT +do_test iptables -F +do_test iptables -N foo +do_test iptables -E foo foo2 +do_test iptables -I OUTPUT -j ACCEPT +do_test iptables -nL +do_test iptables -S + +exit $RC
Signature · 1 line
2.25.1
Phil Sutter
reviewerPATCH v2 18/18 nft: Fix for '-F' in iptables dumps
In reply to Phil SutterWhen restoring a dump which contains an explicit flush command, previously added rules are removed from cache and the following commit will try to create netlink messages based on freed memory.
Fix this by weeding any rule-based commands from obj_list if they address the same chain.
--- iptables/nft.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
iptables/nft.c +34 -0
@@ -402,6 +402,38 @@ batch_rule_add(struct nft_handle *h, enum obj_update_type type,return batch_add (h, type, r); } +static void batch_obj_del (struct nft_handle *h,struct obj_update *o); + +static void batch_chain_flush (struct nft_handle *h, +const char *table,const char *chain) +{ +struct obj_update *obj, *tmp; + +list_for_each_entry_safe (obj, tmp, &h->obj_list, head) { +struct nftnl_rule *r = obj->ptr; + +switch (obj->type) { +case NFT_COMPAT_RULE_APPEND : +case NFT_COMPAT_RULE_INSERT : +case NFT_COMPAT_RULE_REPLACE : +case NFT_COMPAT_RULE_DELETE : +break ; +default : +continue ; + } + +if (table && +strcmp (table,nftnl_rule_get_str (r,NFTNL_RULE_TABLE ))) +continue ; + +if (chain && +strcmp (chain,nftnl_rule_get_str (r,NFTNL_RULE_CHAIN ))) +continue ; + +batch_obj_del (h, obj); + } +} +const struct builtin_table xtables_ipv4[NFT_TABLE_MAX ] = { [NFT_TABLE_RAW ] = { .name ="raw" ,
@@ -1681,6 +1713,7 @@ int nft_rule_flush(struct nft_handle *h, const char *chain, const char *table, }if (chain || !verbose) { +batch_chain_flush (h, table, chain);__nft_rule_flush (h, table, chain, verbose, false);flush_rule_cache (h, table, c);return 1 ;
@@ -1696,6 +1729,7 @@ int nft_rule_flush(struct nft_handle *h, const char *chain, const char *table,while (c !=NULL ) { chain =nftnl_chain_get_str (c,NFTNL_CHAIN_NAME ); +batch_chain_flush (h, table, chain);__nft_rule_flush (h, table, chain, verbose, false);flush_rule_cache (h, table, c); c =nftnl_chain_list_iter_next (iter);
Signature · 1 line
2.25.1
Pablo Neira Ayuso
maintainerHi Phil,
Quoted 5 lines · Phil Sutter
Hi Pablo,
As promised, here's a revised version of your cache rework series from January. It restores performance according to my tests (which are yet to be published somewhere) and passes the testsuites.
I did not test this yet, and I made a few rounds of quick reviews alrady, but this series LGTM. Thank you for working on this.
Phil Sutter
reviewerHi Pablo,
Quoted 9 lines · Pablo Neira Ayuso
Quoted 5 lines · Phil Sutter
Hi Pablo,
As promised, here's a revised version of your cache rework series from January. It restores performance according to my tests (which are yet to be published somewhere) and passes the testsuites.
I did not test this yet, and I made a few rounds of quick reviews alrady, but this series LGTM. Thank you for working on this.
Cool! Should I push it or do you want to have a closer look first?
Cheers, Phil
Pablo Neira Ayuso
maintainerQuoted 6 lines · Phil Sutter
Hi Pablo,
Quoted 3 lines · Pablo Neira Ayuso
Quoted 2 lines · Phil Sutter
Hi Pablo,
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
As promised, here's a revised version of your cache rework series from January. It restores performance according to my tests (which are yet to be published somewhere) and passes the testsuites.
I did not test this yet, and I made a few rounds of quick reviews alrady, but this series LGTM. Thank you for working on this.
Cool! Should I push it or do you want to have a closer look first?
You already took the time to test this, so I think it's fine if you push out. Problems can be fixed from master. It would also good a few runs to valgrind.
BTW, this cache consistency check
commit 200bc399651499f502ac0de45f4d4aa4c9d37ab6
Author: Phil Sutter <[email protected]> Date: Fri Mar 13 13:02:12 2020 +0100
nft: cache: Fix iptables-save segfault under stress
is already restored in this series, right?
Thanks.
Phil Sutter
reviewerHi,
Quoted 11 lines · Pablo Neira Ayuso
Quoted 10 lines · Phil Sutter
Hi Pablo,
Quoted 7 lines · Pablo Neira Ayuso
Quoted 5 lines · Phil Sutter
Hi Pablo,
As promised, here's a revised version of your cache rework series from January. It restores performance according to my tests (which are yet to be published somewhere) and passes the testsuites.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
I did not test this yet, and I made a few rounds of quick reviews alrady, but this series LGTM. Thank you for working on this.
Cool! Should I push it or do you want to have a closer look first?
You already took the time to test this, so I think it's fine if you push out. Problems can be fixed from master. It would also good a few runs to valgrind.
OK, I'll play a bit with valgrind just to be sure and then push it out.
Quoted 9 lines
BTW, this cache consistency check
commit
200bc399651499f502ac0de45f4d4aa4c9d37ab6Author: Phil Sutter <[email protected]> Date: Fri Mar 13 13:02:12 2020 +0100nft: cache: Fix iptables-save segfault under stress
is already restored in this series, right?
Yes, IIRC this was the reason why I got a merge conflict upon rebase. But the problem shouldn't exist with the new logic: We fetch cache just once, so there is no cache update (and potential cache free) happening while iterating through chain lists or anything.
Cheers, Phil
Pablo Neira Ayuso
maintainerOn Thu, Apr 30, 2020 at 05:26:07PM +0200, Phil Sutter wrote:
[...]
Quoted 6 lines
Quoted 6 lines
BTW, this cache consistency check
commit
200bc399651499f502ac0de45f4d4aa4c9d37ab6Author: Phil Sutter <[email protected]> Date: Fri Mar 13 13:02:12 2020 +0100
Quoted 8 lines
Quoted 3 lines
nft: cache: Fix iptables-save segfault under stress
is already restored in this series, right?
Yes, IIRC this was the reason why I got a merge conflict upon rebase. But the problem shouldn't exist with the new logic: We fetch cache just once, so there is no cache update (and potential cache free) happening while iterating through chain lists or anything.
Still another process might be competing to update the ruleset, right?
Pablo Neira Ayuso
maintainerQuoted 10 lines · Pablo Neira Ayuso
On Thu, Apr 30, 2020 at 05:26:07PM +0200, Phil Sutter wrote:
[...]Quoted 8 lines
Quoted 8 lines
BTW, this cache consistency check
commit
200bc399651499f502ac0de45f4d4aa4c9d37ab6Author: Phil Sutter <[email protected]> Date: Fri Mar 13 13:02:12 2020 +0100nft: cache: Fix iptables-save segfault under stress
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
is already restored in this series, right?
Yes, IIRC this was the reason why I got a merge conflict upon rebase. But the problem shouldn't exist with the new logic: We fetch cache just once, so there is no cache update (and potential cache free) happening while iterating through chain lists or anything.
Still another process might be competing to update the ruleset, right?
I mean this case:
- mnl_genid_get(h, &genid_stop);
- if (genid_start != genid_stop) {
- flush_chain_cache(h, NULL);
- goto retry;
- }
if the cache is inconsistent (another process stepped in and updated the ruleset), the discard this cache and fetch it again.
Phil Sutter
reviewerQuoted 28 lines · Pablo Neira Ayuso
Quoted 10 lines · Pablo Neira Ayuso
On Thu, Apr 30, 2020 at 05:26:07PM +0200, Phil Sutter wrote:
[...]Quoted 8 lines
Quoted 8 lines
BTW, this cache consistency check
commit
200bc399651499f502ac0de45f4d4aa4c9d37ab6Author: Phil Sutter <[email protected]> Date: Fri Mar 13 13:02:12 2020 +0100nft: cache: Fix iptables-save segfault under stress
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
is already restored in this series, right?
Yes, IIRC this was the reason why I got a merge conflict upon rebase. But the problem shouldn't exist with the new logic: We fetch cache just once, so there is no cache update (and potential cache free) happening while iterating through chain lists or anything.
Still another process might be competing to update the ruleset, right?
I mean this case:
(quoted diff) +0
-5- mnl_genid_get(h, &genid_stop); - if (genid_start != genid_stop) { - flush_chain_cache(h, NULL); - goto retry; - }
Quoted 2 lines
if the cache is inconsistent (another process stepped in and updated the ruleset), the discard this cache and fetch it again.
Ah, I understand your point now. Let me try to reenable this so __nft_build_cache() returns a cache in which all entries "match" regarding genid. I think this shouldn't cause problems anymore, even if cache is freed in between attempts.
Thanks, Phil