ruleset optimization infrastructure
Pablo Neira Ayuso
maintainerv2 cover ruleset optimization infrastructure
Hi,
This patchset adds a new -o/--optimize option to enable ruleset optimization. Two type of optimizations are supported in this batch:
* Use a set to compact several rules with the same selector using a set, for example:
meta iifname eth1 ip saddr 1.1.1.1 ip daddr 2.2.2.3 accept
meta iifname eth1 ip saddr 1.1.1.2 ip daddr 2.2.2.5 accept
meta iifname eth2 ip saddr 1.1.1.3 ip daddr 2.2.2.6 acceptinto:
meta iifname . ip saddr . ip daddr { eth1 . 1.1.1.1 . 2.2.2.6, eth1 . 1.1.1.2 . 2.2.2.5 , eth1 . 1.1.1.3 . 2.2.2.6 } accept* Use a verdict map to compact rules with same selectors but different verdicts, for example:
ip saddr 1.1.1.1 ip daddr 2.2.2.2 accept
ip saddr 2.2.2.2 ip daddr 3.3.3.3 dropinto:
ip saddr . ip daddr vmap { 1.1.1.1 . 2.2.2.2 : accept, 2.2.2.2 . 3.3.3.3 : drop }Updates since last patch series:
- display information on the rule merges that are proposed, this can be combined with -c to inspect the proposed ruleset updates.
# nft -c -o -f ruleset.nft
This allows sysadmins to review the proposed optimization without actually loading the ruleset, in case they prefer to manually edit their rulesets to apply the proposed optimizations (requested by Arturo).
- tests/py and tests/shell run fine after this new iteration.
- fix error reporting with /dev/stdin, which is a prerequisite for this series.
- fixes.
Pablo Neira Ayuso (7):
erec: expose print_location() and line_location()
src: error reporting with -f and read from stdin
src: remove '$' in symbol_expr_print
src: add ruleset optimization infrastructure
optimize: merge rules with same selectors into a concatenation
optimize: merge same selector with different verdict into verdict map
optimize: merge several selectors with different verdict into verdict map
doc/nft.txt | 5 + include/erec.h | 5 + include/nftables.h | 5 + include/nftables/libnftables.h | 7 + include/rule.h | 1 - src/Makefile.am | 1 + src/erec.c | 87 ++- src/expression.c | 33 +- src/libnftables.c | 109 ++- src/libnftables.map | 5 + src/main.c | 9 +- src/optimize.c | 698 ++++++++++++++++++ src/scanner.l | 2 +- .../optimizations/dumps/merge_stmts.nft | 5 + .../dumps/merge_stmts_concat.nft | 5 + .../dumps/merge_stmts_concat_vmap.nft | 5 + .../optimizations/dumps/merge_stmts_vmap.nft | 5 + .../shell/testcases/optimizations/merge_stmts | 13 + .../optimizations/merge_stmts_concat | 13 + .../optimizations/merge_stmts_concat_vmap | 13 + .../testcases/optimizations/merge_stmts_vmap | 12 + 21 files changed, 992 insertions(+), 46 deletions(-) create mode 100644 src/optimize.c create mode 100644 tests/shell/testcases/optimizations/dumps/merge_stmts.nft create mode 100644 tests/shell/testcases/optimizations/dumps/merge_stmts_concat.nft create mode 100644 tests/shell/testcases/optimizations/dumps/merge_stmts_concat_vmap.nft create mode 100644 tests/shell/testcases/optimizations/dumps/merge_stmts_vmap.nft create mode 100755 tests/shell/testcases/optimizations/merge_stmts create mode 100755 tests/shell/testcases/optimizations/merge_stmts_concat create mode 100755 tests/shell/testcases/optimizations/merge_stmts_concat_vmap create mode 100755 tests/shell/testcases/optimizations/merge_stmts_vmap
Pablo Neira Ayuso
maintainerPATCH v2 1/7 erec: expose print_location() and line_location()
Add a few helper functions to reuse code in the new rule optimization infrastructure.
--- include/erec.h | 5 ++++ include/rule.h | 1 - src/erec.c | 81 +++++++++++++++++++++++++++++++------------------- 3 files changed, 56 insertions(+), 31 deletions(-)
include/erec.h +5 -0
@@ -76,4 +76,9 @@ extern int __fmtstring(4, 5) __stmt_binary_error(struct eval_ctx *ctx, #define stmt_binary_error(ctx, s1, s2, fmt, args...) \ __stmt_binary_error(ctx, &(s1)->location, &(s2)->location, fmt, ## args) +void print_location (FILE *f,const struct input_descriptor *indesc, +const struct location *loc); +const char *line_location (const struct input_descriptor *indesc, +const struct location *loc,char *buf,size_t bufsiz); +#endif /* NFTABLES_EREC_H */
include/rule.h +0 -1
@@ -311,7 +311,6 @@ void rule_stmt_append(struct rule *rule, struct stmt *stmt);void rule_stmt_insert_at (struct rule *rule,struct stmt *nstmt,struct stmt *stmt); -/** * struct set - nftables set *
src/erec.c +51 -30
102 lines not on this page · raw message
Signature · 1 line
2.30.2
Pablo Neira Ayuso
maintainerPATCH v2 2/7 src: error reporting with -f and read from stdin
In reply to Pablo Neira AyusoReading from stdin requires to store the ruleset in a buffer so error reporting works accordingly, eg.
# cat ruleset.nft | nft -f -
/dev/stdin:3:13-13: Error: unknown identifier 'x'
ip saddr $x
^The error reporting infrastructure performs a fseek() on the file descriptor which does not work in this case since the data from the descriptor has been already consumed.
This patch adds a new stdin input descriptor to perform this special handling which consists on re-routing this request through the buffer functions.
--- include/nftables.h | 2 ++ src/erec.c | 8 +++++++- src/libnftables.c | 48 ++++++++++++++++++++++++++++++++++++++++++---- src/scanner.l | 2 +- 4 files changed, 54 insertions(+), 6 deletions(-)
include/nftables.h +2 -0
@@ -128,6 +128,7 @@ struct nft_ctx {struct scope *top_scope;void *json_root;json_t *json_echo; +const char *stdin_buf; };enum nftables_exit_codes {
@@ -175,6 +176,7 @@ enum input_descriptor_types {INDESC_FILE ,INDESC_CLI ,INDESC_NETLINK , + INDESC_STDIN, };/**
src/erec.c +7 -1
@@ -101,10 +101,11 @@ void print_location(FILE *f, const struct input_descriptor *indesc, iloc = &tmp->location; } } -if (indesc->name !=NULL ) +if (indesc->type !=INDESC_BUFFER && indesc->name) {fprintf (f,"%s:%u:%u-%u: " , indesc->name, loc->first_line, loc->first_column, loc->last_column); + } } const char *line_location(const struct input_descriptor *indesc,
@@ -145,6 +146,11 @@ void erec_print(struct output_ctx *octx, const struct error_record *erec, line = indesc->data; *strchrnul (line,'\n' ) ='\0' ;break ; +case INDESC_STDIN : + line = indesc->data; + line += loc->line_offset; + *strchrnul (line,'\n' ) ='\0' ; +break ;case INDESC_FILE : line =line_location (indesc, loc, buf,sizeof (buf));break ;
src/libnftables.c +44 -4
93 lines not on this page · raw message
src/scanner.l +1 -1
@@ -1032,7 +1032,7 @@ void scanner_push_buffer(void *scanner, const struct input_descriptor *indesc, new_indesc = xzalloc(sizeof(struct input_descriptor)); memcpy(new_indesc, indesc, sizeof(*new_indesc)); new_indesc->data = buffer; - new_indesc->name = NULL; + new_indesc->name = xstrdup(indesc->name); scanner_push_indesc(state, new_indesc); b = yy_scan_string(buffer, scanner);
Signature · 1 line
2.30.2
Pablo Neira Ayuso
maintainerPATCH v2 3/7 src: remove '$' in symbol_expr_print
In reply to Pablo Neira AyusoThis is used in --debug=eval mode to annotate symbols that have not yet been evaluated, remove it.
--- src/expression.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
src/expression.c +1 -2
@@ -300,8 +300,7 @@ struct expr *verdict_expr_alloc(const struct location *loc,static void symbol_expr_print (const struct expr *expr,struct output_ctx *octx) { -nft_print (octx,"%s%s" , expr->scope !=NULL ?"$" :"" , - expr->identifier); +nft_print (octx,"%s" , expr->identifier); }static void symbol_expr_clone (struct expr *new,const struct expr *expr)
Signature · 1 line
2.30.2
Pablo Neira Ayuso
maintainerPATCH v2 4/7 src: add ruleset optimization infrastructure
In reply to Pablo Neira AyusoThis patch adds a new -o/--optimize option to enable ruleset optimization.
You can combine this option with the dry run mode (--check) to review the proposed ruleset updates without actually loading the ruleset, e.g.
# nft -c -o -f ruleset.test
Merging:
ruleset.nft:16:3-37: ip daddr 192.168.0.1 counter accept
ruleset.nft:17:3-37: ip daddr 192.168.0.2 counter accept
ruleset.nft:18:3-37: ip daddr 192.168.0.3 counter accept
into:
ip daddr { 192.168.0.1, 192.168.0.2, 192.168.0.3 } counter packets 0 bytes 0 acceptThis infrastructure collects the common statements that are used in rules, then it builds a matrix of rules vs. statements. Then, it looks for common statements in consecutive rules which allows to merge rules.
This ruleset optimization always performs an implicit dry run to validate that the original ruleset is correct. Then, on a second pass, it performs the ruleset optimization and add the rules into the kernel (unless --check has been specified by the user).
From libnftables perspective, there is a new API to enable this feature:
uint32_t nft_ctx_get_optimize(struct nft_ctx *ctx); void nft_ctx_set_optimize(struct nft_ctx *ctx, uint32_t flags);
This patch adds support for the first optimization: Collapse a linear list of rules matching on a single selector into a set as exposed in the example above.
--- doc/nft.txt | 5 + include/nftables.h | 3 + include/nftables/libnftables.h | 7 + src/Makefile.am | 1 + src/libnftables.c | 71 ++- src/libnftables.map | 5 + src/main.c | 9 +- src/optimize.c | 473 ++++++++++++++++++ .../optimizations/dumps/merge_stmts.nft | 5 + .../shell/testcases/optimizations/merge_stmts | 13 + 10 files changed, 581 insertions(+), 11 deletions(-) create mode 100644 src/optimize.c create mode 100644 tests/shell/testcases/optimizations/dumps/merge_stmts.nft create mode 100755 tests/shell/testcases/optimizations/merge_stmts
doc/nft.txt +5 -0
@@ -62,6 +62,11 @@ understanding of their meaning. You can get information about options by running *--check*:: Check commands validity without actually applying the changes. +*-o*:: +*--optimize*:: + Optimize your ruleset. You can combine this option with '-c' to inspect + the proposed optimizations. + .Ruleset list output formatting that modify the output of the list ruleset command: *-a*::
include/nftables.h +3 -0
@@ -123,6 +123,7 @@ struct nft_ctx {bool check;struct nft_cache cache;uint32_t flags; +uint32_t optimize_flags;struct parser_state *state;void *scanner;struct scope *top_scope;
@@ -224,6 +225,8 @@ int nft_print(struct output_ctx *octx, const char *fmt, ...)int nft_gmp_print (struct output_ctx *octx,const char *fmt, ...) __attribute__((format (printf,2 ,0 ))); +int nft_optimize (struct nft_ctx *nft,struct list_head *cmds); +#define __NFT_OUTPUT_NOTSUPP UINT_MAX#endif /* NFTABLES_NFTABLES_H */
include/nftables/libnftables.h +7 -0
@@ -41,6 +41,13 @@ void nft_ctx_free(struct nft_ctx *ctx);bool nft_ctx_get_dry_run (struct nft_ctx *ctx);void nft_ctx_set_dry_run (struct nft_ctx *ctx,bool dry); +enum nft_optimize_flags { +NFT_OPTIMIZE_ENABLED =0x1 , +}; + +uint32_t nft_ctx_get_optimize (struct nft_ctx *ctx); +void nft_ctx_set_optimize (struct nft_ctx *ctx,uint32_t flags); +enum {NFT_CTX_OUTPUT_REVERSEDNS = (1 <<0 ),NFT_CTX_OUTPUT_SERVICE = (1 <<1 ),
src/Makefile.am +1 -0
@@ -70,6 +70,7 @@ libnftables_la_SOURCES = \ mnl.c \ iface.c \ mergesort.c \ + optimize.c \ osf.c \ nfnl_osf.c \ tcpopt.c \
src/libnftables.c +61 -10
101 lines not on this page · raw message
src/libnftables.map +5 -0
@@ -28,3 +28,8 @@ LIBNFTABLES_2 { nft_ctx_add_var; nft_ctx_clear_vars; } LIBNFTABLES_1; + +LIBNFTABLES_3 { + nft_set_optimize; + nft_get_optimize; +} LIBNFTABLES_2;
src/main.c +8 -1
@@ -36,7 +36,8 @@ enum opt_indices {IDX_INTERACTIVE ,IDX_INCLUDEPATH ,IDX_CHECK , -#define IDX_RULESET_INPUT_END IDX_CHECK +IDX_OPTIMIZE , +#define IDX_RULESET_INPUT_END IDX_OPTIMIZE /* Ruleset list formatting */ IDX_HANDLE,#define IDX_RULESET_LIST_START IDX_HANDLE
@@ -80,6 +81,7 @@ enum opt_vals {OPT_NUMERIC_PROTO ='p' ,OPT_NUMERIC_TIME ='T' ,OPT_TERSE ='t' , +OPT_OPTIMIZE ='o' , OPT_INVALID = '?', };
@@ -136,6 +138,8 @@ static const struct nft_opt nft_options[] = { "Format output in JSON"), [IDX_DEBUG ] =NFT_OPT ("debug" ,OPT_DEBUG ,"<level [,level...]>" , "Specify debugging level (scanner, parser, eval, netlink, mnl, proto-ctx, segtree, all)"), + [IDX_OPTIMIZE ] =NFT_OPT ("optimize" ,OPT_OPTIMIZE ,NULL , + "Optimize ruleset"), };#define NR_NFT_OPTIONS (sizeof(nft_options) / sizeof(nft_options[0]))
@@ -484,6 +488,9 @@ int main(int argc, char * const *argv)case OPT_TERSE : output_flags |=NFT_CTX_OUTPUT_TERSE ;break ; +case OPT_OPTIMIZE : +nft_ctx_set_optimize (nft,0x1 ); +break ;case OPT_INVALID :exit (EXIT_FAILURE ); }
src/optimize.c new file +473 -0
473 lines not on this page · raw message
tests/shell/testcases/optimizations/dumps/merge_stmts.nft new file +5 -0
@@ -0,0 +1,5 @@ +table ip x { + chain y { + ip daddr { 192.168.0.1, 192.168.0.2, 192.168.0.3 } counter packets 0 bytes 0 accept + } +}
tests/shell/testcases/optimizations/merge_stmts new file +13 -0
@@ -0,0 +1,13 @@ +#!/bin/bash + +set -e + +RULESET="table ip x { + chain y { + ip daddr 192.168.0.1 counter accept + ip daddr 192.168.0.2 counter accept + ip daddr 192.168.0.3 counter accept + } +}" + +$NFT -o -f - <<< $RULESET
Signature · 1 line
2.30.2
Pablo Neira Ayuso
maintainerPATCH v2 5/7 optimize: merge rules with same selectors into a concatenation
In reply to Pablo Neira AyusoThis patch extends the ruleset optimization infrastructure to collapse several rules with the same selectors into a concatenation.
Transform:
meta iifname eth1 ip saddr 1.1.1.1 ip daddr 2.2.2.3 accept meta iifname eth1 ip saddr 1.1.1.2 ip daddr 2.2.2.5 accept meta iifname eth2 ip saddr 1.1.1.3 ip daddr 2.2.2.6 accept
into:
meta iifname . ip saddr . ip daddr { eth1 . 1.1.1.1 . 2.2.2.6, eth1 . 1.1.1.2 . 2.2.2.5 , eth1 . 1.1.1.3 . 2.2.2.6 } accept
--- src/optimize.c | 44 ++++++++++++++++++- .../dumps/merge_stmts_concat.nft | 5 +++ .../optimizations/merge_stmts_concat | 13 ++++++ 3 files changed, 61 insertions(+), 1 deletion(-) create mode 100644 tests/shell/testcases/optimizations/dumps/merge_stmts_concat.nft create mode 100755 tests/shell/testcases/optimizations/merge_stmts_concat
src/optimize.c +43 -1
@@ -266,6 +266,48 @@ static void merge_stmts(const struct optimize_ctx *ctx, stmt_a->expr->right = set; } +static void merge_concat_stmts (const struct optimize_ctx *ctx, +uint32_t from,uint32_t to, +const struct merge *merge) +{ +struct expr *concat, *elem, *set; +struct stmt *stmt, *stmt_a; +uint32_t i, k; + + stmt = ctx->stmt_matrix[from][merge->stmt[0 ]]; +/* build concatenation of selectors, eg. ifname . ip daddr . tcp dport */ + concat =concat_expr_alloc (&internal_location); + +for (k =0 ; k < merge->num_stmts; k++) { + stmt_a = ctx->stmt_matrix[from][merge->stmt[k]]; +compound_expr_add (concat,expr_get (stmt_a->expr->left)); + } +expr_free (stmt->expr->left); + stmt->expr->left = concat; + +/* build set data contenation, eg. { eth0 . 1.1.1.1 . 22 } */ + set =set_expr_alloc (&internal_location,NULL ); + set->set_flags |=NFT_SET_ANONYMOUS ; + +for (i = from; i <= to; i++) { + concat =concat_expr_alloc (&internal_location); +for (k =0 ; k < merge->num_stmts; k++) { + stmt_a = ctx->stmt_matrix[i][merge->stmt[k]]; +compound_expr_add (concat,expr_get (stmt_a->expr->right)); + } + elem =set_elem_expr_alloc (&internal_location, concat); +compound_expr_add (set, elem); + } +expr_free (stmt->expr->right); + stmt->expr->right = set; + +for (k =1 ; k < merge->num_stmts; k++) { + stmt_a = ctx->stmt_matrix[from][merge->stmt[k]]; +list_del (&stmt_a->list); +stmt_free (stmt_a); + } +} +static void rule_optimize_print (struct output_ctx *octx,const struct rule *rule) {
@@ -307,7 +349,7 @@ static void merge_rules(const struct optimize_ctx *ctx,uint32_t i;if (merge->num_stmts >1 ) { -return ; +merge_concat_stmts (ctx, from, to, merge); }else {merge_stmts (ctx, from, to, merge); }
tests/shell/testcases/optimizations/dumps/merge_stmts_concat.nft new file +5 -0
@@ -0,0 +1,5 @@ +table ip x { + chain y { + iifname . ip saddr . ip daddr { "eth1" . 1.1.1.1 . 2.2.2.3, "eth1" . 1.1.1.2 . 2.2.2.4, "eth2" . 1.1.1.3 . 2.2.2.5 } accept + } +}
tests/shell/testcases/optimizations/merge_stmts_concat new file +13 -0
@@ -0,0 +1,13 @@ +#!/bin/bash + +set -e + +RULESET="table ip x { + chain y { + meta iifname eth1 ip saddr 1.1.1.1 ip daddr 2.2.2.3 accept + meta iifname eth1 ip saddr 1.1.1.2 ip daddr 2.2.2.4 accept + meta iifname eth2 ip saddr 1.1.1.3 ip daddr 2.2.2.5 accept + } +}" + +$NFT -o -f - <<< $RULESET
Signature · 1 line
2.30.2
Pablo Neira Ayuso
maintainerPATCH v2 6/7 optimize: merge same selector with different verdict into verdict map
In reply to Pablo Neira AyusoTransform:
ct state invalid drop
ct state established,related accept
into:
ct state vmap { established : accept, related : accept, invalid : drop }
--- src/optimize.c | 162 ++++++++++++++++-- .../optimizations/dumps/merge_stmts_vmap.nft | 5 + .../testcases/optimizations/merge_stmts_vmap | 12 ++ 3 files changed, 164 insertions(+), 15 deletions(-) create mode 100644 tests/shell/testcases/optimizations/dumps/merge_stmts_vmap.nft create mode 100755 tests/shell/testcases/optimizations/merge_stmts_vmap
src/optimize.c +147 -15
@@ -90,19 +90,6 @@ static bool __stmt_type_eq(const struct stmt *stmt_a, const struct stmt *stmt_b)case STMT_NOTRACK :break ;case STMT_VERDICT : - expr_a = stmt_a->expr; - expr_b = stmt_b->expr; -if (expr_a->verdict != expr_b->verdict) -return false; -if (expr_a->chain && expr_b->chain) { -if (expr_a->chain->etype != expr_b->chain->etype) -return false; -if (expr_a->chain->etype ==EXPR_VALUE && -strcmp (expr_a->chain->identifier, expr_b->chain->identifier)) -return false; - }else if (expr_a->chain || expr_b->chain) { -return false; - }break ;case STMT_LIMIT : if (stmt_a->limit.rate != stmt_b->limit.rate ||
@@ -141,6 +128,29 @@ static bool __stmt_type_eq(const struct stmt *stmt_a, const struct stmt *stmt_b)return true; } +static bool stmt_verdict_eq (const struct stmt *stmt_a,const struct stmt *stmt_b) +{ +struct expr *expr_a, *expr_b; + +assert (stmt_a->ops->type ==STMT_VERDICT ); + + expr_a = stmt_a->expr; + expr_b = stmt_b->expr; +if (expr_a->verdict != expr_b->verdict) +return false; +if (expr_a->chain && expr_b->chain) { +if (expr_a->chain->etype != expr_b->chain->etype) +return false; +if (expr_a->chain->etype ==EXPR_VALUE && +strcmp (expr_a->chain->identifier, expr_b->chain->identifier)) +return false; + }else if (expr_a->chain || expr_b->chain) { +return false; + } + +return true; +} +static bool stmt_type_eq (const struct stmt *stmt_a,const struct stmt *stmt_b) {if (!stmt_a && !stmt_b)
@@ -229,6 +239,20 @@ static void rule_build_stmt_matrix_stmts(struct optimize_ctx *ctx, ctx->rule[(*i)++] = rule; } +static int stmt_verdict_find (const struct optimize_ctx *ctx) +{ +uint32_t i; + +for (i =0 ; i < ctx->num_stmts; i++) { +if (ctx->stmt[i]->ops->type !=STMT_VERDICT ) +continue ; + +return i; + } + +return -1 ; +} +struct merge {/* interval of rules to be merged */ uint32_t rule_from;
@@ -308,6 +332,105 @@ static void merge_concat_stmts(const struct optimize_ctx *ctx, } } +static void build_verdict_map (struct expr *expr,struct stmt *verdict,struct expr *set) +{ +struct expr *item, *elem, *mapping; + +if (expr->etype ==EXPR_LIST ) { +list_for_each_entry (item, &expr->expressions, list) { + elem =set_elem_expr_alloc (&internal_location,expr_get (item)); + mapping =mapping_expr_alloc (&internal_location, elem, +expr_get (verdict->expr)); +compound_expr_add (set, mapping); + } + }else { + elem =set_elem_expr_alloc (&internal_location,expr_get (expr)); + mapping =mapping_expr_alloc (&internal_location, elem, +expr_get (verdict->expr)); +compound_expr_add (set, mapping); + } +} + +static void remove_counter (const struct optimize_ctx *ctx,uint32_t from) +{ +struct stmt *stmt; +uint32_t i; + +/* remove counter statement */ +for (i =0 ; i < ctx->num_stmts; i++) { + stmt = ctx->stmt_matrix[from][i]; +if (!stmt) +continue ; + +if (stmt->ops->type ==STMT_COUNTER ) { +list_del (&stmt->list); +stmt_free (stmt); + } + } +} + +static void merge_stmts_vmap (const struct optimize_ctx *ctx, +uint32_t from,uint32_t to, +const struct merge *merge) +{ +struct stmt *stmt_a = ctx->stmt_matrix[from][merge->stmt[0 ]]; +struct stmt *stmt_b, *verdict_a, *verdict_b, *stmt; +struct expr *expr_a, *expr_b, *expr, *left, *set; +uint32_t i; +int k; + + k =stmt_verdict_find (ctx); +assert (k >=0 ); + + verdict_a = ctx->stmt_matrix[from][k]; + set =set_expr_alloc (&internal_location,NULL ); + set->set_flags |=NFT_SET_ANONYMOUS ; + + expr_a = stmt_a->expr->right; +build_verdict_map (expr_a, verdict_a, set); +for (i = from +1 ; i <= to; i++) { + stmt_b = ctx->stmt_matrix[i][merge->stmt[0 ]]; + expr_b = stmt_b->expr->right; + verdict_b = ctx->stmt_matrix[i][k]; + +build_verdict_map (expr_b, verdict_b, set); + } + + left =expr_get (stmt_a->expr->left); + expr =map_expr_alloc (&internal_location, left, set); + stmt =verdict_stmt_alloc (&internal_location, expr); + +remove_counter (ctx, from); +list_add (&stmt->list, &stmt_a->list); +list_del (&stmt_a->list); +stmt_free (stmt_a); +list_del (&verdict_a->list); +stmt_free (verdict_a); +} + +static bool stmt_verdict_cmp (const struct optimize_ctx *ctx, +uint32_t from,uint32_t to) +{ +struct stmt *stmt_a, *stmt_b; +uint32_t i; +int k; + + k =stmt_verdict_find (ctx); +if (k <0 ) +return true; + +for (i = from; i +1 <= to; i++) { + stmt_a = ctx->stmt_matrix[i][k]; + stmt_b = ctx->stmt_matrix[i +1 ][k]; +if (!stmt_a && !stmt_b) +return true; +if (stmt_verdict_eq (stmt_a, stmt_b)) +return true; + } + +return false; +} +static void rule_optimize_print (struct output_ctx *octx,const struct rule *rule) {
@@ -346,12 +469,21 @@ static void merge_rules(const struct optimize_ctx *ctx,const struct merge *merge, struct output_ctx *octx) { + bool same_verdict;uint32_t i; + same_verdict =stmt_verdict_cmp (ctx, from, to); +if (merge->num_stmts >1 ) { -merge_concat_stmts (ctx, from, to, merge); +if (same_verdict) +merge_concat_stmts (ctx, from, to, merge); +else +return ; }else { -merge_stmts (ctx, from, to, merge); +if (same_verdict) +merge_stmts (ctx, from, to, merge); +else +merge_stmts_vmap (ctx, from, to, merge); }fprintf (octx->error_fp,"Merging:\n" );
tests/shell/testcases/optimizations/dumps/merge_stmts_vmap.nft new file +5 -0
@@ -0,0 +1,5 @@ +table ip x { + chain y { + ct state vmap { invalid : drop, established : accept, related : accept } + } +}
tests/shell/testcases/optimizations/merge_stmts_vmap new file +12 -0
@@ -0,0 +1,12 @@ +#!/bin/bash + +set -e + +RULESET="table ip x { + chain y { + ct state invalid drop + ct state established,related accept + } +}" + +$NFT -o -f - <<< $RULESET
Signature · 1 line
2.30.2
Pablo Neira Ayuso
maintainerPATCH v2 7/7 optimize: merge several selectors with different verdict into verdict map
In reply to Pablo Neira AyusoTransform:
ip saddr 1.1.1.1 ip daddr 2.2.2.2 accept
ip saddr 2.2.2.2 ip daddr 3.3.3.3 drop
into:
ip saddr . ip daddr vmap { 1.1.1.1 . 2.2.2.2 : accept, 2.2.2.2 . 3.3.3.3 : drop }
--- src/expression.c | 30 +++++++++- src/optimize.c | 57 ++++++++++++++++++- .../dumps/merge_stmts_concat_vmap.nft | 5 ++ .../optimizations/merge_stmts_concat_vmap | 13 +++++ 4 files changed, 102 insertions(+), 3 deletions(-) create mode 100644 tests/shell/testcases/optimizations/dumps/merge_stmts_concat_vmap.nft create mode 100755 tests/shell/testcases/optimizations/merge_stmts_concat_vmap
src/expression.c +28 -2
@@ -1185,14 +1185,40 @@ struct expr *mapping_expr_alloc(const struct location *loc,return expr; } +static bool __set_expr_is_vmap (const struct expr *mappings) +{ +const struct expr *mapping; + +if (list_empty (&mappings->expressions)) +return false; + + mapping = list_first_entry(&mappings->expressions, struct expr, list); +if (mapping->etype ==EXPR_MAPPING && + mapping->right->etype ==EXPR_VERDICT ) +return true; + +return false; +} + +static bool set_expr_is_vmap (const struct expr *expr) +{ + +if (expr->mappings->etype ==EXPR_SET ) +return __set_expr_is_vmap (expr->mappings); + +return false; +} +static void map_expr_print (const struct expr *expr,struct output_ctx *octx) {expr_print (expr->map, octx); -if (expr->mappings->etype ==EXPR_SET_REF && - expr->mappings->set->data->dtype->type ==TYPE_VERDICT ) +if ((expr->mappings->etype ==EXPR_SET_REF && + expr->mappings->set->data->dtype->type ==TYPE_VERDICT ) || +set_expr_is_vmap (expr))nft_print (octx," vmap " );else nft_print (octx," map " ); +expr_print (expr->mappings, octx); }
src/optimize.c +56 -1
69 lines not on this page · raw message
tests/shell/testcases/optimizations/dumps/merge_stmts_concat_vmap.nft new file +5 -0
@@ -0,0 +1,5 @@ +table ip x { + chain y { + ip saddr . ip daddr vmap { 1.1.1.1 . 2.2.2.2 : accept, 2.2.2.2 . 3.3.3.3 : drop, 4.4.4.4 . 5.5.5.5 : accept } + } +}
tests/shell/testcases/optimizations/merge_stmts_concat_vmap new file +13 -0
@@ -0,0 +1,13 @@ +#!/bin/bash + +set -e + +RULESET="table ip x { + chain y { + ip saddr 1.1.1.1 ip daddr 2.2.2.2 accept + ip saddr 2.2.2.2 ip daddr 3.3.3.3 drop + ip saddr 4.4.4.4 ip daddr 5.5.5.5 accept + } +}" + +$NFT -o -f - <<< $RULESET
Signature · 1 line
2.30.2
Pablo Neira Ayuso
maintainerOn Sun, Jan 02, 2022 at 11:14:45PM +0100, Pablo Neira Ayuso wrote:
[...]
Quoted 6 lines
Updates since last patch series:
- display information on the rule merges that are proposed, this can be combined with -c to inspect the proposed ruleset updates.
# nft -c -o -f ruleset.nft
For the record, an example output of -c -o
nft -o -c -f ruleset.nft
Merging:
ruleset.nft:3:3-46: ip daddr 192.168.1.0/24 ct state new counter
ruleset.nft:4:3-46: ip daddr 192.168.2.0/24 ct state new counter
ruleset.nft:5:3-46: ip daddr 192.168.3.0/24 ct state new counter
ruleset.nft:6:3-46: ip daddr 192.168.4.0/24 ct state new counter
into:
ip daddr . ct state { 192.168.1.0/24 . new, 192.168.2.0/24 . new, 192.168.3.0/24 . new, 192.168.4.0/24 . new } counter packets 0 bytes 0Merging:
ruleset.nft:7:3-23: ct state invalid drop
ruleset.nft:8:3-37: ct state established,related accept
into:
ct state vmap { invalid : drop, established : accept, related : accept }
Merging:
ruleset.nft:9:3-60: meta iifname eth1 ip saddr 1.1.1.2 ip daddr 2.2.2.3 accept
ruleset.nft:10:3-60: meta iifname eth1 ip saddr 2.2.2.2 ip daddr 2.2.2.5 accept
ruleset.nft:11:3-60: meta iifname eth2 ip saddr 1.1.1.3 ip daddr 2.2.2.6 accept
into:
ip daddr . iifname . ip saddr { 2.2.2.3 . eth1 . 1.1.1.2, 2.2.2.5 . eth1 . 2.2.2.2, 2.2.2.6 . eth2 . 1.1.1.3 } acceptMerging:
ruleset.nft:12:3-97: ip saddr 10.69.0.0/24 ct state new counter packets 0 bytes 0 log prefix "unexpected traffic" level debug
ruleset.nft:13:3-97: ip saddr 10.69.1.0/24 ct state new counter packets 0 bytes 0 log prefix "unexpected traffic" level debug
into:
ct state . ip saddr { new . 10.69.0.0/24, new . 10.69.1.0/24 } counter packets 0 bytes 0 log prefix "unexpected traffic" level debug
Merging:
ruleset.nft:16:3-37: ip daddr 192.168.0.1 counter accept
ruleset.nft:17:3-37: ip daddr 192.168.0.2 counter accept
ruleset.nft:18:3-37: ip daddr 192.168.0.3 counter accept
into:
ip daddr { 192.168.0.1, 192.168.0.2, 192.168.0.3 } counter packets 0 bytes 0 acceptLe 02/01/2022 à 23:14, Pablo Neira Ayuso a écrit :
Quoted 8 lines
Hi,
This patchset adds a new -o/--optimize option to enable ruleset optimization. Two type of optimizations are supported in this batch:
* Use a set to compact several rules with the same selector using a set, for example:
Quoted 7 lines
meta iifname eth1 ip saddr 1.1.1.1 ip daddr 2.2.2.3 accept meta iifname eth1 ip saddr 1.1.1.2 ip daddr 2.2.2.5 accept meta iifname eth2 ip saddr 1.1.1.3 ip daddr 2.2.2.6 acceptinto: meta iifname . ip saddr . ip daddr { eth1 . 1.1.1.1 . 2.2.2.6, eth1 . 1.1.1.2 . 2.2.2.5 , eth1 . 1.1.1.3 . 2.2.2.6 } accept
nit: it would probably be better with this result ;-)
meta iifname . ip saddr . ip daddr { eth1 . 1.1.1.1 . 2.2.2.3, eth1 . 1.1.1.2 . 2.2.2.5 , eth2 . 1.1.1.3 . 2.2.2.6 } accept
Regards,
Nicolas
Pablo Neira Ayuso
maintainerQuoted 4 lines · Pablo Neira Ayuso
Hi,
This patchset adds a new -o/--optimize option to enable ruleset optimization.
For the record, I have pushed out these.