rule: collapse set element commands
Pablo Neira Ayuso
maintainerPATCH v2 1/2 rule: collapse set element commands
Robots might generate a long list of singleton element commands such as:
add element t s { 1.0.1.0/24 }
...
add element t s { 1.0.2.0/23 }
collapse them into one single command before the evaluation step, ie.
add element t s { 1.0.1.0/24, ..., 1.0.2.0/23 }
this speeds up overlap detection and set element automerge operations in this worst case scenario.
Since 3da9643fb9ff9 ("intervals: add support to automerge with kernel elements"), the new interval tracking relies on mergesort. The pattern above triggers the set sorting for each element.
This patch adds a list to cmd objects that store collapsed commands. Moreover, expressions also contain a reference to the original command, to uncollapse the commands after the evaluation step.
These commands are uncollapsed after the evaluation step to ensure error reporting works as expected (command and netlink message are mapped 1:1).
For the record:
- nftables versions <= 1.0.2 did not perform any kind of overlap check for the described scenario above (because set cache only contained elements in the kernel in this case). This is a problem for kernels < 5.7 which rely on userspace to detect overlaps.
- the overlap detection could be skipped for kernels >= 5.7.
- The extended netlink error reporting available for set elements since 5.19-rc might allow to remove the uncollapse step, in this case, error reporting does not rely on the netlink sequence to refer to the command triggering the problem.
---
@Phil: This is v2, restoring error reporting after the command collapse
operation as you mentioned in your feedback.
Note nftables <= 1.0.2 does not perform any sort of overlap checking from userspace for the scenario described in this commit, that's why first nft -f call is "so fast" (set cache was empty in that case).
include/expression.h | 1 + include/rule.h | 3 ++ src/libnftables.c | 17 ++++++++-- src/rule.c | 75 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 93 insertions(+), 3 deletions(-)
include/expression.h +1 -0
@@ -243,6 +243,7 @@ struct expr { enum expr_types etype:8; enum ops op:8;unsigned int len; +struct cmd *cmd; union { struct {
include/rule.h +3 -0
@@ -700,6 +700,7 @@ struct cmd {enum cmd_obj obj;struct handle handle;uint32_t seqnum; +struct list_head collapse_list;union {void *data;struct expr *expr;
@@ -728,6 +729,8 @@ extern struct cmd *cmd_alloc(enum cmd_ops op, enum cmd_obj obj, const struct handle *h, const struct location *loc, void *data);extern void nft_cmd_expand (struct cmd *cmd); +extern bool nft_cmd_collapse (struct list_head *cmds); +extern void nft_cmd_uncollapse (struct list_head *cmds);extern struct cmd *cmd_alloc_obj_ct (enum cmd_ops op,int type,const struct handle *h,const struct location *loc,struct obj *obj);
src/libnftables.c +14 -3
@@ -501,7 +501,9 @@ static int nft_evaluate(struct nft_ctx *nft, struct list_head *msgs, {struct nft_cache_filter *filter;struct cmd *cmd, *next; +bool collapsed = false;unsigned int flags; +int err =0 ; filter =nft_cache_filter_init (); flags =nft_cache_evaluate (nft, cmds, filter);
@@ -512,17 +514,26 @@ static int nft_evaluate(struct nft_ctx *nft, struct list_head *msgs,nft_cache_filter_fini (filter); +if (nft_cmd_collapse (cmds)) + collapsed = true; +list_for_each_entry_safe (cmd, next, cmds, list) {struct eval_ctx ectx = { .nft = nft, .msgs = msgs, }; +if (cmd_evaluate (&ectx, cmd) <0 && - ++nft->state->nerrs == nft->parser_max_errors) -return -1 ; + ++nft->state->nerrs == nft->parser_max_errors) { + err =-1 ; +break ; + } } -if (nft->state->nerrs) +if (collapsed) +nft_cmd_uncollapse (cmds); + +if (err <0 || nft->state->nerrs)return -1 ;list_for_each_entry (cmd, cmds, list) {
src/rule.c +75 -0
87 lines not on this page · raw message
Signature · 1 line
2.30.2
Quoted 32 lines · Pablo Neira Ayuso
Robots might generate a long list of singleton element commands such as:
add element t s { 1.0.1.0/24 }
...
add element t s { 1.0.2.0/23 }collapse them into one single command before the evaluation step, ie.
add element t s { 1.0.1.0/24, ..., 1.0.2.0/23 }
this speeds up overlap detection and set element automerge operations in this worst case scenario.
Since
3da9643fb9ff9("intervals: add support to automerge with kernel elements"), the new interval tracking relies on mergesort. The pattern above triggers the set sorting for each element.This patch adds a list to cmd objects that store collapsed commands. Moreover, expressions also contain a reference to the original command, to uncollapse the commands after the evaluation step.
These commands are uncollapsed after the evaluation step to ensure error reporting works as expected (command and netlink message are mapped 1:1).
For the record:
- nftables versions <= 1.0.2 did not perform any kind of overlap check for the described scenario above (because set cache only contained elements in the kernel in this case). This is a problem for kernels < 5.7 which rely on userspace to detect overlaps.
Quoted 8 lines
- the overlap detection could be skipped for kernels >= 5.7.
- The extended netlink error reporting available for set elements since 5.19-rc might allow to remove the uncollapse step, in this case, error reporting does not rely on the netlink sequence to refer to the command triggering the problem.
Signed-off-by Pablo Neira Ayuso <[email protected]>