iptables: introduce cache evaluation phase
Pablo Neira Ayuso
maintainercover iptables: introduce cache evaluation phase
Hi Phil,
Happy new year.
This patchset introduces a new step to parse rules into a list of commands:
input -> parser -> list of commands -> list of jobs -> netlink -> kernel
This allows us to estimate the cache requirements from the list of commands and to avoid in-transit cache cancelations while handling a batch.
This batch is likely disabling your optimizations where selective chain/set netlink dump speeds up things. From reading the code, I'm not sure what paths are specifically benefiting from this optimization since any of the existing nftnl_chain_list_get() and nftnl_set_list_get() calls might exercise this selective netlink dump. I think it should not be too hard to restore those, I remember you have a few tests to evaluate the speed up. Note that this patch also disables the pre-parsing from xtables-restore.
Downside is that there's an extra memory allocation, that could be consolidated in the future, by having one single list of commands/jobs. But would probably make this batch even larger, but I tried to reduce complexity as much as possible. Debugging cache issue is hard in my opinion, if we can avoid fetch/cancel/fetch cache scenario, the better.
The approach that 'among' follows to use the set infrastructure is interesting, probably there's a way to simplify this, but I understand there might be more urgent stuff to have a look.
Let me know,
Thanks.
P.S: I started this patchset by Dec 25 while traveling, tests look
OK here, I'm just releasing this because I'm not sure I can come back to this anytime soon.
Pablo Neira Ayuso (7):
nft: do not check for existing chain from parser
nft: split parsing from netlink commands
nft: calculate cache requirements from list of commands
nft: restore among support
nft: remove cache build calls
nft: skip table list release if uninitialized
nft: missing nft_fini() call in bridge family
iptables/Makefile.am | 2 +- iptables/nft-arp.c | 5 +- iptables/nft-bridge.c | 18 +- iptables/nft-cache.c | 72 ++-- iptables/nft-cache.h | 1 + iptables/nft-cmd.c | 389 +++++++++++++++++++++ iptables/nft-cmd.h | 79 +++++ iptables/nft-shared.c | 6 +- iptables/nft-shared.h | 4 +- iptables/nft.c | 310 +++++++++++----- iptables/nft.h | 49 ++- .../shell/testcases/ip6tables/0004-return-codes_0 | 2 +- .../shell/testcases/iptables/0004-return-codes_0 | 2 +- iptables/xtables-arp.c | 26 +- iptables/xtables-eb-standalone.c | 2 + iptables/xtables-eb.c | 26 +- iptables/xtables-restore.c | 35 +- iptables/xtables-save.c | 3 + iptables/xtables.c | 57 ++- 19 files changed, 863 insertions(+), 225 deletions(-) create mode 100644 iptables/nft-cmd.c create mode 100644 iptables/nft-cmd.h
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 1/7 nft: do not check for existing chain from parser
Follow up patches split the parser from the cache calculation.
--- iptables/tests/shell/testcases/ip6tables/0004-return-codes_0 | 2 +- iptables/tests/shell/testcases/iptables/0004-return-codes_0 | 2 +- iptables/xtables.c | 5 ----- 3 files changed, 2 insertions(+), 7 deletions(-)
iptables/tests/shell/testcases/ip6tables/0004-return-codes_0 +1 -1
@@ -31,7 +31,7 @@ cmd 1 ip6tables -A noexist -j ACCEPT cmd 0 ip6tables -C INPUT -j ACCEPT cmd 1 ip6tables -C FORWARD -j ACCEPT cmd 1 ip6tables -C nonexist -j ACCEPT -cmd 2 ip6tables -C INPUT -j foobar +cmd 1 ip6tables -C INPUT -j foobar cmd 2 ip6tables -C INPUT -m foobar -j ACCEPT cmd 3 ip6tables -t foobar -C INPUT -j ACCEPT
iptables/tests/shell/testcases/iptables/0004-return-codes_0 +1 -1
@@ -75,7 +75,7 @@ cmd 2 "$ENOMTH" iptables -C INPUT -m foobar -j ACCEPT # messages of those don't match, but iptables-nft ones are actually nicer. #cmd 2 "$ENOTGT" iptables -C INPUT -j foobar #cmd 3 "$ENOTBL" iptables -t foobar -C INPUT -j ACCEPT -cmd 2 "" iptables -C INPUT -j foobar +cmd 1 "" iptables -C INPUT -j foobar cmd 3 "" iptables -t foobar -C INPUT -j ACCEPT exit $global_rc
iptables/xtables.c +0 -5
@@ -1031,11 +1031,6 @@ void do_parse(struct nft_handle *h, int argc, char *argv[],opt2char (OPT_VIANAMEIN ), p->chain); } - -if (!p->xlate && !cs->target &&strlen (cs->jumpto) >0 && - !nft_chain_exists (h, p->table, cs->jumpto)) -xtables_error (PARAMETER_PROBLEM , -"Chain '%s' does not exist" , cs->jumpto); } }
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 2/7 nft: split parsing from netlink commands
In reply to Pablo Neira AyusoThis patch updates the parser to generate a list of command objects. This list of commands is then transformed to a list of netlink jobs. This new command object stores the rule using the nftnl representation via nft_rule_new().
To reduce the number of updates in this patch, the nft_*_rule_find() functions have been updated to restore the native representation to skip the update of the rule comparison code.
--- iptables/Makefile.am | 2 +- iptables/nft-arp.c | 5 +- iptables/nft-bridge.c | 5 +- iptables/nft-cmd.c | 315 +++++++++++++++++++++++++++++++++++++++++++++ iptables/nft-cmd.h | 78 +++++++++++ iptables/nft-shared.c | 6 +- iptables/nft-shared.h | 4 +- iptables/nft.c | 259 +++++++++++++++++++++++++++---------- iptables/nft.h | 41 +++++- iptables/xtables-arp.c | 26 ++-- iptables/xtables-eb.c | 26 ++-- iptables/xtables-restore.c | 32 ++--- iptables/xtables.c | 52 ++++---- 13 files changed, 702 insertions(+), 149 deletions(-) create mode 100644 iptables/nft-cmd.c create mode 100644 iptables/nft-cmd.h
iptables/Makefile.am +1 -1
@@ -38,7 +38,7 @@ xtables_nft_multi_SOURCES += xtables-save.c xtables-restore.c \ nft-shared.c nft-ipv4.c nft-ipv6.c nft-arp.c \ xtables-monitor.c nft-cache.c \ xtables-arp-standalone.c xtables-arp.c \ - nft-bridge.c \ + nft-bridge.c nft-cmd.c \ xtables-eb-standalone.c xtables-eb.c \ xtables-eb-translate.c \ xtables-translate.c
iptables/nft-arp.c +3 -2
@@ -634,14 +634,15 @@ static bool nft_arp_is_same(const void *data_a, }static bool nft_arp_rule_find (struct nft_handle *h,struct nftnl_rule *r, -void *data) +struct nftnl_rule *rule) { -const struct iptables_command_state *cs = data; +struct iptables_command_state _cs = {}, *cs = &_cs;struct iptables_command_state this = {};bool ret = false;/* Delete by matching rule case */ nft_rule_to_iptables_command_state (h, r, &this); +nft_rule_to_iptables_command_state (h, rule, cs);if (!nft_arp_is_same (&cs->arp, &this.arp)) goto out;
iptables/nft-bridge.c +3 -2
@@ -748,13 +748,14 @@ static bool nft_bridge_is_same(const void *data_a, const void *data_b) }static bool nft_bridge_rule_find (struct nft_handle *h,struct nftnl_rule *r, -void *data) +struct nftnl_rule *rule) { -struct iptables_command_state *cs = data; +struct iptables_command_state _cs = {}, *cs = &_cs;struct iptables_command_state this = {};bool ret = false;nft_rule_to_ebtables_command_state (h, r, &this); +nft_rule_to_ebtables_command_state (h, rule, cs);DEBUGP ("comparing with... " );
iptables/nft-cmd.c new file +315 -0
315 lines not on this page · raw message
iptables/nft-cmd.h new file +78 -0
78 lines not on this page · raw message
iptables/nft-shared.c +4 -2
@@ -989,12 +989,14 @@ void nft_ipv46_parse_target(struct xtables_target *t, void *data) cs->target = t; } -bool nft_ipv46_rule_find (struct nft_handle *h,struct nftnl_rule *r,void *data) +bool nft_ipv46_rule_find (struct nft_handle *h,struct nftnl_rule *r, +struct nftnl_rule *rule) { -struct iptables_command_state *cs = data, this = {}; +struct iptables_command_state _cs = {}, this = {}, *cs = &_cs;bool ret = false;nft_rule_to_iptables_command_state (h, r, &this); +nft_rule_to_iptables_command_state (h, rule, cs);DEBUGP ("comparing with... " );#ifdef DEBUG_DEL
iptables/nft-shared.h +2 -2
@@ -110,7 +110,7 @@ struct nft_family_ops { struct iptables_command_state *cs);void (*clear_cs)(struct iptables_command_state *cs);bool (*rule_find)(struct nft_handle *h,struct nftnl_rule *r, -void *data); +struct nftnl_rule *rule);int (*xlate)(const void *data,struct xt_xlate *xl); };
@@ -172,7 +172,7 @@ struct nft_family_ops *nft_family_ops_lookup(int family);void nft_ipv46_parse_target (struct xtables_target *t,void *data);bool nft_ipv46_rule_find (struct nft_handle *h,struct nftnl_rule *r, -void *data); +struct nftnl_rule *rule);bool compare_matches (struct xtables_rule_match *mt1,struct xtables_rule_match *mt2);bool compare_targets (struct xtables_target *tg1,struct xtables_target *tg2);
iptables/nft.c +193 -66
485 lines not on this page · raw message
iptables/nft.h +35 -6
72 lines not on this page · raw message
iptables/xtables-arp.c +13 -13
@@ -400,7 +400,7 @@ list_entries(struct nft_handle *h, const char *chain, const char *table,if (linenumbers) format |=FMT_LINENUMBERS ; -return nft_rule_list (h, chain, table, rulenum, format); +return nft_cmd_rule_list (h, chain, table, rulenum, format); } static int
@@ -427,10 +427,10 @@ append_entry(struct nft_handle *h, cs->arp.arp.tgt.s_addr = daddrs[j].s_addr; cs->arp.arp.tmsk.s_addr = dmasks[j].s_addr;if (append) { - ret =nft_rule_append (h, chain, table, cs,NULL , + ret =nft_cmd_rule_append (h, chain, table, cs,NULL , verbose); }else { - ret =nft_rule_insert (h, chain, table, cs, + ret =nft_cmd_rule_insert (h, chain, table, cs, rulenum, verbose); } }
@@ -455,7 +455,7 @@ replace_entry(const char *chain, cs->arp.arp.smsk.s_addr = smask->s_addr; cs->arp.arp.tmsk.s_addr = dmask->s_addr; -return nft_rule_replace (h, chain, table, cs, rulenum, verbose); +return nft_cmd_rule_replace (h, chain, table, cs, rulenum, verbose); } static int
@@ -479,7 +479,7 @@ delete_entry(const char *chain,for (j =0 ; j < ndaddrs; j++) { cs->arp.arp.tgt.s_addr = daddrs[j].s_addr; cs->arp.arp.tmsk.s_addr = dmasks[j].s_addr; - ret =nft_rule_delete (h, chain, table, cs, verbose); + ret =nft_cmd_rule_delete (h, chain, table, cs, verbose); } }
@@ -962,7 +962,7 @@ int do_commandarp(struct nft_handle *h, int argc, char *argv[], char **table, options&OPT_VERBOSE, h);break ;case CMD_DELETE_NUM : - ret =nft_rule_delete_num (h, chain, *table, rulenum -1 , verbose); + ret =nft_cmd_rule_delete_num (h, chain, *table, rulenum -1 , verbose);break ;case CMD_REPLACE : ret = replace_entry(chain, *table, &cs, rulenum - 1,
@@ -984,10 +984,10 @@ int do_commandarp(struct nft_handle *h, int argc, char *argv[], char **table, options&OPT_LINENUMBERS);break ;case CMD_FLUSH : - ret =nft_rule_flush (h, chain, *table, options &OPT_VERBOSE ); + ret =nft_cmd_rule_flush (h, chain, *table, options &OPT_VERBOSE );break ;case CMD_ZERO : - ret =nft_chain_zero_counters (h, chain, *table, + ret =nft_cmd_chain_zero_counters (h, chain, *table, options &OPT_VERBOSE );break ;case CMD_LIST |CMD_ZERO :
@@ -997,21 +997,21 @@ int do_commandarp(struct nft_handle *h, int argc, char *argv[], char **table,/*options&OPT_EXPANDED*/ 0 , options&OPT_LINENUMBERS);if (ret) - ret =nft_chain_zero_counters (h, chain, *table, + ret =nft_cmd_chain_zero_counters (h, chain, *table, options &OPT_VERBOSE );break ;case CMD_NEW_CHAIN : - ret =nft_chain_user_add (h, chain, *table); + ret =nft_cmd_chain_user_add (h, chain, *table);break ;case CMD_DELETE_CHAIN : - ret =nft_chain_user_del (h, chain, *table, + ret =nft_cmd_chain_user_del (h, chain, *table, options &OPT_VERBOSE );break ;case CMD_RENAME_CHAIN : - ret =nft_chain_user_rename (h, chain, *table, newname); + ret =nft_cmd_chain_user_rename (h, chain, *table, newname);break ;case CMD_SET_POLICY : - ret =nft_chain_set (h, *table, chain, policy,NULL ); + ret =nft_cmd_chain_set (h, *table, chain, policy,NULL );if (ret <0 )xtables_error (PARAMETER_PROBLEM ,"Wrong policy `%s'\n" , policy);
iptables/xtables-eb.c +13 -13
@@ -150,9 +150,9 @@ append_entry(struct nft_handle *h,int ret =1 ;if (append) - ret =nft_rule_append (h, chain, table, cs,NULL , verbose); + ret =nft_cmd_rule_append (h, chain, table, cs,NULL , verbose);else - ret =nft_rule_insert (h, chain, table, cs, rule_nr, verbose); + ret =nft_cmd_rule_insert (h, chain, table, cs, rule_nr, verbose);return ret; }
@@ -169,10 +169,10 @@ delete_entry(struct nft_handle *h,int ret =1 ;if (rule_nr ==-1 ) - ret =nft_rule_delete (h, chain, table, cs, verbose); + ret =nft_cmd_rule_delete (h, chain, table, cs, verbose);else {do { - ret =nft_rule_delete_num (h, chain, table, + ret =nft_cmd_rule_delete_num (h, chain, table, rule_nr, verbose); rule_nr++; }while (rule_nr < rule_nr_end);
@@ -427,7 +427,7 @@ static int list_rules(struct nft_handle *h, const char *chain, const char *tableif (!counters) format |=FMT_NOCOUNTS ; -return nft_rule_list (h, chain, table, rule_nr, format); +return nft_cmd_rule_list (h, chain, table, rule_nr, format); }static int parse_rule_range (const char *argv,int *rule_nr,int *rule_nr_end)
@@ -820,7 +820,7 @@ int do_commandeb(struct nft_handle *h, int argc, char *argv[], char **table, flags |=OPT_COMMAND ;if (c =='N' ) { - ret =nft_chain_user_add (h, chain, *table); + ret =nft_cmd_chain_user_add (h, chain, *table);break ; }else if (c =='X' ) {/* X arg is optional, optarg is NULL */
@@ -828,7 +828,7 @@ int do_commandeb(struct nft_handle *h, int argc, char *argv[], char **table, chain = argv[optind]; optind++; } - ret =nft_chain_user_del (h, chain, *table,0 ); + ret =nft_cmd_chain_user_del (h, chain, *table,0 );break ; }
@@ -842,7 +842,7 @@ int do_commandeb(struct nft_handle *h, int argc, char *argv[], char **table,else if (strchr (argv[optind],' ' ) !=NULL )xtables_error (PARAMETER_PROBLEM ,"Use of ' ' not allowed in chain names" ); - ret =nft_chain_user_rename (h, chain, *table, + ret =nft_cmd_chain_user_rename (h, chain, *table, argv[optind]);if (ret !=0 && errno ==ENOENT )xtables_error (PARAMETER_PROBLEM ,"Chain '%s' doesn't exists" , chain);
@@ -1144,7 +1144,7 @@ print_zero:/*case 7 :*/ /* atomic-init */ /*case 10:*/ /* atomic-save */ case 11 :/* init-table */ -nft_table_flush (h, *table); +nft_cmd_table_flush (h, *table);return 1 ;/* replace->command = c;
@@ -1232,13 +1232,13 @@ print_zero:if (command =='P' ) {if (selected_chain >=NF_BR_NUMHOOKS ) { - ret =ebt_set_user_chain_policy (h, *table, chain, policy); + ret =ebt_cmd_user_chain_policy (h, *table, chain, policy); }else {if (strcmp (policy,"RETURN" ) ==0 ) {xtables_error (PARAMETER_PROBLEM ,"Policy RETURN only allowed for user defined chains" ); } - ret =nft_chain_set (h, *table, chain, policy,NULL ); + ret =nft_cmd_chain_set (h, *table, chain, policy,NULL );if (ret <0 )xtables_error (PARAMETER_PROBLEM ,"Wrong policy" ); }
@@ -1251,9 +1251,9 @@ print_zero: flags&LIST_C); }if (flags &OPT_ZERO ) { - ret =nft_chain_zero_counters (h, chain, *table,0 ); + ret =nft_cmd_chain_zero_counters (h, chain, *table,0 ); }else if (command =='F' ) { - ret =nft_rule_flush (h, chain, *table,0 ); + ret =nft_cmd_rule_flush (h, chain, *table,0 ); }else if (command =='A' ) { ret =append_entry (h, chain, *table, &cs,0 ,0 , true); }else if (command =='I' ) {
iptables/xtables-restore.c +16 -16
@@ -61,11 +61,11 @@ static void print_usage(const char *name, const char *version)static const struct nft_xt_restore_cb restore_cb = { .commit = nft_commit, .abort = nft_abort, - .table_new = nft_table_new, - .table_flush = nft_table_flush, + .table_new = nft_cmd_table_new, + .table_flush = nft_cmd_table_flush, .do_command = do_commandx, - .chain_set = nft_chain_set, - .chain_restore = nft_chain_restore, + .chain_set = nft_cmd_chain_set, + .chain_restore = nft_cmd_chain_restore, };struct nft_xt_restore_state {
@@ -189,7 +189,7 @@ static void xtables_restore_parse_line(struct nft_handle *h,"cannot create chain '%s' (%s)\n" , chain, strerror(errno)); }else if (h->family ==NFPROTO_BRIDGE && - !ebt_set_user_chain_policy (h, state->curtable->name, + !ebt_cmd_user_chain_policy (h, state->curtable->name, chain, policy)) { xtables_error(OTHER_PROBLEM, "Can't set policy `%s' on `%s' line %u: %s\n",
@@ -489,20 +489,20 @@ int xtables_ip6_restore_main(int argc, char *argv[]) argc, argv); } -static int ebt_table_flush (struct nft_handle *h,const char *table) +static int ebt_cmd_table_flush (struct nft_handle *h,const char *table) {/* drop any pending policy rule add/removal jobs */ -nft_abort_policy_rule (h, table); -return nft_table_flush (h, table); +nft_cmd_abort_policy_rule (h, table); +return nft_cmd_table_flush (h, table); }static const struct nft_xt_restore_cb ebt_restore_cb = { .commit = nft_bridge_commit, - .table_new = nft_table_new, - .table_flush = ebt_table_flush, + .table_new = nft_cmd_table_new, + .table_flush = ebt_cmd_table_flush, .do_command = do_commandeb, - .chain_set = nft_chain_set, - .chain_restore = nft_chain_restore, + .chain_set = nft_cmd_chain_set, + .chain_restore = nft_cmd_chain_restore, };static const struct option ebt_restore_options[] = {
@@ -544,11 +544,11 @@ int xtables_eb_restore_main(int argc, char *argv[])static const struct nft_xt_restore_cb arp_restore_cb = { .commit = nft_commit, - .table_new = nft_table_new, - .table_flush = nft_table_flush, + .table_new = nft_cmd_table_new, + .table_flush = nft_cmd_table_flush, .do_command = do_commandarp, - .chain_set = nft_chain_set, - .chain_restore = nft_chain_restore, + .chain_set = nft_cmd_chain_set, + .chain_restore = nft_cmd_chain_restore, };int xtables_arp_restore_main (int argc,char *argv[])
iptables/xtables.c +26 -26
158 lines not on this page · raw message
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 3/7 nft: calculate cache requirements from list of commands
In reply to Pablo Neira AyusoThis patch uses the new list of commands to calculate the cache requirements, the rationale after this updates is the following:
#1 Parsing, that builds the list of commands and it also calculates
cache level requirements.
#2 Cache building.
#3 Translate commands to jobs
#4 Translate jobs to netlink
This patch disables the pre-parsing code in xtables-restore.c to calculate the cache.
After this patch, cache is calculated only once, there is no need to cancel and refetch for an in-transit transaction.
--- iptables/nft-cache.c | 60 +++++++++++++++++-------------------- iptables/nft-cache.h | 1 + iptables/nft-cmd.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++ iptables/nft-cmd.h | 1 + iptables/nft.c | 17 +++++++++-- iptables/nft.h | 2 +- iptables/xtables-restore.c | 3 -- iptables/xtables-save.c | 3 ++ 8 files changed, 122 insertions(+), 39 deletions(-)
iptables/nft-cache.c +27 -33
@@ -454,36 +454,31 @@ __nft_build_cache(struct nft_handle *h, enum nft_cache_level level, {uint32_t genid_start, genid_stop; -if (level <= h->cache_level) +if (h->cache_init)return ; + + h->cache_init = true; retry:mnl_genid_get (h, &genid_start); -if (h->cache_level && genid_start != h->nft_genid) -flush_chain_cache (h,NULL ); -switch (h->cache_level) { -case NFT_CL_NONE : -fetch_table_cache (h); -if (level ==NFT_CL_TABLES ) -break ; -/* fall through */ case NFT_CL_TABLES : -fetch_chain_cache (h, t, chain); -if (level ==NFT_CL_CHAINS ) -break ; -/* fall through */ +fetch_table_cache (h); +break ;case NFT_CL_CHAINS : -fetch_set_cache (h, t, set); -if (level ==NFT_CL_SETS ) -break ; -/* fall through */ +fetch_table_cache (h); +fetch_chain_cache (h, t, chain); +break ;case NFT_CL_SETS : -fetch_rule_cache (h, t, chain); -if (level ==NFT_CL_RULES ) -break ; -/* fall through */ +fetch_table_cache (h); +fetch_chain_cache (h, t, chain); +fetch_set_cache (h, t, set); +break ;case NFT_CL_RULES : +fetch_table_cache (h); +fetch_chain_cache (h, t, chain); +fetch_set_cache (h, t, set); +fetch_rule_cache (h, t, chain);break ; }
@@ -493,11 +488,6 @@ retry: goto retry; } -if (!t && !chain) - h->cache_level = level; -else if (h->cache_level <NFT_CL_TABLES ) - h->cache_level =NFT_CL_TABLES ; - h->nft_genid = genid_start; }
@@ -529,6 +519,7 @@ void nft_fake_cache(struct nft_handle *h) h->cache->table[type].chains =nftnl_chain_list_alloc (); } h->cache_level =NFT_CL_RULES ; + h->cache_init = true;mnl_genid_get (h, &h->nft_genid); }
@@ -627,22 +618,25 @@ static int flush_cache(struct nft_handle *h, struct nft_cache *c,void flush_chain_cache (struct nft_handle *h,const char *tablename) { -if (!h->cache_level) +if (!h->cache_init)return ;if (flush_cache (h, h->cache, tablename)) - h->cache_level =NFT_CL_NONE ; + h->cache_init = false; }void nft_rebuild_cache (struct nft_handle *h) { -enum nft_cache_level level = h->cache_level; - -if (h->cache_level) +if (h->cache_init) {__nft_flush_cache (h); + h->cache_init = false; + } +__nft_build_cache (h, h->cache_level,NULL ,NULL ,NULL ); +} - h->cache_level =NFT_CL_NONE ; -__nft_build_cache (h, level,NULL ,NULL ,NULL ); +void nft_cache_build (struct nft_handle *h) +{ +__nft_build_cache (h, h->cache_level,NULL ,NULL ,NULL ); }void nft_release_cache (struct nft_handle *h)
iptables/nft-cache.h +1 -0
@@ -10,6 +10,7 @@ void nft_release_cache(struct nft_handle *h);void flush_chain_cache (struct nft_handle *h,const char *tablename);int flush_rule_cache (struct nft_handle *h,const char *table,struct nftnl_chain *c); +void nft_cache_build (struct nft_handle *h);struct nftnl_chain_list *nft_chain_list_get (struct nft_handle *h,const char *table,const char *chain);
iptables/nft-cmd.c +74 -0
193 lines not on this page · raw message
iptables/nft-cmd.h +1 -0
@@ -18,6 +18,7 @@ struct nft_cmd {unsigned int format;struct {struct nftnl_rule *rule; +struct nftnl_set *set; } obj;const char *policy;struct xt_counters counters;
iptables/nft.c +15 -2
@@ -947,6 +947,7 @@ static struct nftnl_set *add_anon_set(struct nft_handle *h, const char *table, {static uint32_t set_id =0 ;struct nftnl_set *s; +struct nft_cmd *cmd; s =nftnl_set_alloc ();if (!s)
@@ -962,7 +963,14 @@ static struct nftnl_set *add_anon_set(struct nft_handle *h, const char *table,nftnl_set_set_u32 (s,NFTNL_SET_KEY_LEN , key_len);nftnl_set_set_u32 (s,NFTNL_SET_DESC_SIZE , size); -return batch_set_add (h,NFT_COMPAT_SET_ADD , s) ? s :NULL ; + cmd =nft_cmd_new (h,NFT_COMPAT_SET_ADD , table,NULL ,NULL ,-1 , false); +if (!cmd) { +nftnl_set_free (s); +return NULL ; + } + cmd->obj.set = s; + +return s; } static struct nftnl_expr *
@@ -2964,6 +2972,8 @@ static int nft_prepare(struct nft_handle *h)struct nft_cmd *cmd, *next;int ret =1 ; +nft_cache_build (h); +list_for_each_entry_safe (cmd, next, &h->cmd_list, head) {switch (cmd->command) {case NFT_COMPAT_TABLE_FLUSH :
@@ -3044,9 +3054,12 @@ static int nft_prepare(struct nft_handle *h)nft_xt_builtin_init (h, cmd->table); ret =1 ;break ; +case NFT_COMPAT_SET_ADD : +batch_set_add (h,NFT_COMPAT_SET_ADD , cmd->obj.set); + ret =1 ; +break ;case NFT_COMPAT_TABLE_ADD :case NFT_COMPAT_CHAIN_ADD : -case NFT_COMPAT_SET_ADD :assert (0 );break ; }
iptables/nft.h +1 -1
@@ -30,7 +30,6 @@ struct builtin_table { };enum nft_cache_level { -NFT_CL_NONE ,NFT_CL_TABLES ,NFT_CL_CHAINS ,NFT_CL_SETS ,
@@ -94,6 +93,7 @@ struct nft_handle {bool noflush;int8_t config_done;struct list_head cmd_list; +bool cache_init;/* meta data, for error reporting */ struct {
iptables/xtables-restore.c +0 -3
@@ -319,9 +319,6 @@ void xtables_restore_parse(struct nft_handle *h, ptr += blen; buffer[0 ] ='\0' ; } - -if (do_cache) -nft_build_cache (h,NULL ); } line =0 ;
iptables/xtables-save.c +3 -0
@@ -253,6 +253,9 @@ xtables_save_main(int family, int argc, char *argv[],if (!h.ops)xtables_error (PARAMETER_PROBLEM ,"Unknown family" ); + h.cache_level =NFT_CL_RULES ; +nft_cache_build (&h); + ret =do_output (&h, tablename, &d);nft_fini (&h);if (dump)
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 4/7 nft: restore among support
In reply to Pablo Neira AyusoUpdate among support to work again with the new parser and cache logic.
--- iptables/nft-bridge.c | 13 +++++++++++-- iptables/nft.c | 15 +++++++++++++++ iptables/nft.h | 6 ++++++ 3 files changed, 32 insertions(+), 2 deletions(-)
iptables/nft-bridge.c +11 -2
@@ -421,11 +421,20 @@ static struct nftnl_set *set_from_lookup_expr(struct nft_xt_ctx *ctx, const struct nftnl_expr *e) {const char *set_name =nftnl_expr_get_str (e,NFTNL_EXPR_LOOKUP_SET ); +uint32_t set_id =nftnl_expr_get_u32 (e,NFTNL_EXPR_LOOKUP_SET_ID );struct nftnl_set_list *slist; +struct nftnl_set *set; slist =nft_set_list_get (ctx->h, ctx->table, set_name); -if (slist) -return nftnl_set_list_lookup_byname (slist, set_name); +if (slist) { + set =nftnl_set_list_lookup_byname (slist, set_name); +if (set) +return set; + + set =nft_set_batch_lookup_byid (ctx->h, set_id); +if (set) +return set; + }return NULL ; }
iptables/nft.c +15 -0
@@ -1588,6 +1588,20 @@ int nft_rule_save(struct nft_handle *h, const char *table, unsigned int format)return ret ==0 ?1 :0 ; } +struct nftnl_set *nft_set_batch_lookup_byid (struct nft_handle *h, +uint32_t set_id) +{ +struct obj_update *n; + +list_for_each_entry (n, &h->obj_list, head) { +if (n->type ==NFT_COMPAT_SET_ADD && +nftnl_set_get_u32 (n->set,NFTNL_SET_ID ) == set_id) +return n->set; + } + +return NULL ; +} +static void __nft_rule_flush (struct nft_handle *h,const char *table,const char *chain,bool verbose,bool implicit)
@@ -3055,6 +3069,7 @@ static int nft_prepare(struct nft_handle *h) ret =1 ;break ;case NFT_COMPAT_SET_ADD : +nft_xt_builtin_init (h, cmd->table);batch_set_add (h,NFT_COMPAT_SET_ADD , cmd->obj.set); ret =1 ;break ;
iptables/nft.h +6 -0
@@ -144,6 +144,12 @@ void nft_bridge_chain_postprocess(struct nft_handle *h,/* +* Operations with sets. +*/ +struct nftnl_set *nft_set_batch_lookup_byid (struct nft_handle *h, +uint32_t set_id); + +/* * Operations with rule-set. */ struct nftnl_rule ;
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 5/7 nft: remove cache build calls
In reply to Pablo Neira AyusoThe cache requirements are now calculated once from the parsing phase. There is no need to call __nft_build_cache() from several spots in the codepath anymore.
--- iptables/nft-cache.c | 6 ------ iptables/nft.c | 21 --------------------- 2 files changed, 27 deletions(-)
iptables/nft-cache.c +0 -6
@@ -647,8 +647,6 @@ void nft_release_cache(struct nft_handle *h)struct nftnl_table_list *nftnl_table_list_get (struct nft_handle *h) { -__nft_build_cache (h,NFT_CL_TABLES ,NULL ,NULL ,NULL ); -return h->cache->tables; }
@@ -661,8 +659,6 @@ nft_set_list_get(struct nft_handle *h, const char *table, const char *set)if (!t)return NULL ; -__nft_build_cache (h,NFT_CL_RULES , t, set,NULL ); -return h->cache->table[t->type].sets; }
@@ -675,8 +671,6 @@ nft_chain_list_get(struct nft_handle *h, const char *table, const char *chain)if (!t)return NULL ; -__nft_build_cache (h,NFT_CL_CHAINS , t,NULL , chain); -return h->cache->table[t->type].chains; }
iptables/nft.c +0 -21
@@ -1342,14 +1342,6 @@ nft_rule_append(struct nft_handle *h, const char *chain, const char *table,nft_xt_builtin_init (h, table); -/* Since ebtables user-defined chain policies are implemented as last -* rule in nftables, rule cache is required here to treat them right. */ -if (h->family ==NFPROTO_BRIDGE ) { - c =nft_chain_find (h, table, chain); -if (c && !nft_chain_builtin (c)) -nft_build_cache (h, c); - } - nft_fn = nft_rule_append;if (ref) {
@@ -1574,7 +1566,6 @@ int nft_rule_save(struct nft_handle *h, const char *table, unsigned int format) c =nftnl_chain_list_iter_next (iter);while (c) { -nft_build_cache (h, c); ret =nft_chain_save_rules (h, c, format);if (ret !=0 )break ;
@@ -1782,10 +1773,6 @@ static int __nft_chain_user_del(struct nftnl_chain *c, void *data)fprintf (stdout,"Deleting chain `%s'\n" ,nftnl_chain_get_str (c,NFTNL_CHAIN_NAME )); -/* This triggers required policy rule deletion. */ -if (h->family ==NFPROTO_BRIDGE ) -nft_build_cache (h, c); -/* XXX This triggers a fast lookup from the kernel. */ nftnl_chain_unset (c,NFTNL_CHAIN_HANDLE ); ret =batch_chain_add (h,NFT_COMPAT_CHAIN_USER_DEL , c);
@@ -2068,8 +2055,6 @@ nft_rule_find(struct nft_handle *h, struct nftnl_chain *c,struct nftnl_rule_iter *iter;bool found = false; -nft_build_cache (h, c); -if (rulenum >=0 )/* Delete by rule number case */ return nftnl_rule_lookup_byindex (c, rulenum);
@@ -2955,8 +2940,6 @@ int ebt_set_user_chain_policy(struct nft_handle *h, const char *table,else return 0 ; -nft_build_cache (h, c); -nftnl_chain_set_u32 (c,NFTNL_CHAIN_POLICY , pval);return 1 ; }
@@ -3317,8 +3300,6 @@ static int __nft_chain_zero_counters(struct nftnl_chain *c, void *data)return -1 ; } -nft_build_cache (h, c); - iter =nftnl_rule_iter_create (c);if (iter ==NULL )return -1 ;
@@ -3455,8 +3436,6 @@ static int nft_is_chain_compatible(struct nftnl_chain *c, void *data)enum nf_inet_hooks hook;int prio; -nft_build_cache (h, c); -if (nftnl_rule_foreach (c, nft_is_rule_compatible,NULL ))return -1 ;
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 6/7 nft: skip table list release if uninitialized
In reply to Pablo Neira Ayuso# printf '%s\nCOMMIT\n' '*nat' '*raw' '*filter' | iptables-nft-restore --test && echo ok Segmentation fault
--- iptables/nft-cache.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
iptables/nft-cache.c +4 -2
@@ -610,8 +610,10 @@ static int flush_cache(struct nft_handle *h, struct nft_cache *c,nftnl_set_list_free (c->table[i].sets); c->table[i].sets =NULL ; } -nftnl_table_list_free (c->tables); - c->tables =NULL ; +if (c->tables) { +nftnl_table_list_free (c->tables); + c->tables =NULL ; + }return 1 ; }
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 7/7 nft: missing nft_fini() call in bridge family
In reply to Pablo Neira Ayuso--- iptables/xtables-eb-standalone.c | 2 ++ 1 file changed, 2 insertions(+)
iptables/xtables-eb-standalone.c +2 -0
@@ -53,6 +53,8 @@ int xtables_eb_main(int argc, char *argv[])if (ret) ret =nft_bridge_commit (&h); +nft_fini (&h); +if (!ret)fprintf (stderr,"ebtables: %s\n" ,nft_strerror (errno));
Signature · 1 line
2.11.0