xt: Implement dump and restore support
Phil Sutter
reviewerv2 cover xt: Implement dump and restore support
If nft can't translate a compat expression, dump it in a format that can be restored later without losing data, thereby keeping the ruleset intact.
Patch 1 is preparation (more or less), patch 2 has the gory details, patch 3 is a minor code refactoring that's almost unrelated and patch 4 further sanitizes behaviour now that there's a reliable fallback in place.
Changes since v1:
- Use patch 3 to also improve the error printing if extension lookup fails.
- New patch 4.
Phil Sutter (4):
xt: Delay libxtables access until translation
xt: Implement dump and restore support
xt: Put match/target translation into own functions
xt: Detect xlate callback failure
configure.ac | 12 +- doc/libnftables-json.adoc | 15 +- doc/statements.txt | 17 ++ include/base64.h | 17 ++ include/json.h | 2 + include/parser.h | 1 + include/statement.h | 9 +- include/xt.h | 4 + src/Makefile.am | 3 +- src/base64.c | 170 ++++++++++++++++++++ src/evaluate.c | 1 + src/json.c | 25 ++- src/netlink_linearize.c | 32 ++++ src/parser_bison.y | 28 ++++ src/parser_json.c | 36 +++++ src/scanner.l | 14 ++ src/statement.c | 1 + src/xt.c | 317 ++++++++++++++++++++++---------------- 18 files changed, 558 insertions(+), 146 deletions(-) create mode 100644 include/base64.h create mode 100644 src/base64.c
Signature · 1 line
2.38.0
Phil Sutter
reviewerPATCH v2 1/4 xt: Delay libxtables access until translation
There is no point in spending efforts setting up the xt match/target when it is not printed afterwards. So just store the statement data from libnftnl in struct xt_stmt and perform the extension lookup from xt_stmt_xlate() instead.
This means some data structures are only temporarily allocated for the sake of passing to libxtables callbacks, no need to drag them around. Also no need to clone the looked up extension, it is needed only to call the functions it provides.
While being at it, select numeric output in xt_xlate_*_params - otherwise there will be reverse DNS lookups which should not happen by default.
--- include/statement.h | 9 +-- src/xt.c | 192 ++++++++++++++++++-------------------------- 2 files changed, 80 insertions(+), 121 deletions(-)
include/statement.h +4 -5
@@ -264,12 +264,11 @@ struct xtables_target;struct xt_stmt {const char *name;enum nft_xt_type type; +uint32_t rev; +uint32_t family; +size_t infolen; +void *info;uint32_t proto; -union { -struct xtables_match *match; -struct xtables_target *target; - }; -void *entry; };extern struct stmt *xt_stmt_alloc (const struct location *loc);
src/xt.c +76 -116
278 lines not on this page · raw message
Signature · 1 line
2.38.0
Phil Sutter
reviewerPATCH v2 2/4 xt: Implement dump and restore support
In reply to Phil SutterWhen listing a rule with compat expressions, fall back to a textual representation which does not require knowledge of the underlying data structures (i.e. libxtables support), preserves information and gives some clue to users what it is.
The statement is printed with its type and name in clear text, the remaining private data and revision encoded as base64 string. To prevent listings from becoming a total mess, the data is zipped before encoding (if zlib is available).
Base64 en-/decoding implementation copied from FreeBSD and slightly adjusted to fit our needs.
--- configure.ac | 12 ++- doc/libnftables-json.adoc | 15 +++- doc/statements.txt | 17 ++++ include/base64.h | 17 ++++ include/json.h | 2 + include/parser.h | 1 + include/xt.h | 4 + src/Makefile.am | 3 +- src/base64.c | 170 ++++++++++++++++++++++++++++++++++++++ src/evaluate.c | 1 + src/json.c | 25 ++++-- src/netlink_linearize.c | 32 +++++++ src/parser_bison.y | 28 +++++++ src/parser_json.c | 36 ++++++++ src/scanner.l | 14 ++++ src/statement.c | 1 + src/xt.c | 99 ++++++++++++++++++++-- 17 files changed, 458 insertions(+), 19 deletions(-) create mode 100644 include/base64.h create mode 100644 src/base64.c
configure.ac +11 -1
@@ -98,6 +98,15 @@ AC_DEFINE([HAVE_LIBXTABLES], [1], [0]) ]) AM_CONDITIONAL([BUILD_XTABLES], [test "x$with_xtables" = xyes]) +AC_ARG_WITH([zlib], [AS_HELP_STRING([--without-zlib], + [Disable payload compression of xt compat expressions when listing])], + [], [with_zlib=yes]) +AS_IF([test "x$with_zlib" != xno], [ +AC_CHECK_LIB([z], [compress], , + AC_MSG_ERROR([No suitable version of zlib found])) +AC_DEFINE([HAVE_ZLIB], [1], [Define if you have zlib]) +]) + AC_ARG_WITH([json], [AS_HELP_STRING([--with-json], [Enable JSON output support])], [], [with_json=no])
@@ -156,7 +165,8 @@ echo " use mini-gmp: ${with_mini_gmp} enable man page: ${enable_man_doc} libxtables support: ${with_xtables} - json output support: ${with_json}" + json output support: ${with_json} + compress xt compat payloads: ${with_zlib}" AS_IF([test "$enable_python" != "no"], [ echo " enable Python: yes (with $PYTHON_BIN)"
doc/libnftables-json.adoc +12 -3
@@ -1059,10 +1059,19 @@ Assign connection tracking expectation. === XT [verse] -*{ "xt": null }* +____ +*{ "xt": [* 'TYPE' *,* 'STRING' *,* 'STRING' *] }* + +'TYPE' := *match* | *target* | *watcher* +____ + +This represents an xt statement from xtables compat interface. It merely exists +to support saving and restoring a ruleset containing these statements without +losing data. -This represents an xt statement from xtables compat interface. Sadly, at this -point, it is not possible to provide any further information about its content. +The textual representation was chosen to give users a rough idea of what it is +by explicitly stating the extension's 'TYPE' and name in the first two array +fields. The third one embeds the extension's data in compressed base64 format. == EXPRESSIONS Expressions are the building blocks of (most) statements. In their most basic
doc/statements.txt +17 -0
@@ -783,3 +783,20 @@ ____ # jump to different chains depending on layer 4 protocol type: nft add rule ip filter input ip protocol vmap { tcp : jump tcp-chain, udp : jump udp-chain , icmp : jump icmp-chain } ------------------------ + +XT STATEMENT +~~~~~~~~~~~~ +This represents an xt statement from xtables compat interface. It merely exists +to support saving and restoring a ruleset containing these statements without +losing data. + +[verse] +____ +*xt* 'TYPE' 'NAME' 'BASE64' + +'TYPE' := *match* | *target* | *watcher* +____ + +The textual representation was chosen to give users a rough idea of what it is +by explicitly stating the extension's 'TYPE' and 'NAME' followed by the +extension's data in compressed 'BASE64' format.
include/base64.h new file +17 -0
@@ -0,0 +1,17 @@ +/* +* Base64 encoding/decoding (RFC1341) +* Copyright (c) 2005, Jouni Malinen <[email protected]> +* +* This software may be distributed under the terms of the BSD license. +* See README for more details. +*/ + +#ifndef BASE64_H +#define BASE64_H + +unsigned char *base64_encode (const unsigned char *src,size_t len, +size_t *out_len); +unsigned char *base64_decode (const unsigned char *src,size_t len, +size_t *out_len); + +#endif /* BASE64_H */
include/json.h +2 -0
@@ -92,6 +92,7 @@ json_t *connlimit_stmt_json(const struct stmt *stmt, struct output_ctx *octx);json_t *tproxy_stmt_json (const struct stmt *stmt,struct output_ctx *octx);json_t *synproxy_stmt_json (const struct stmt *stmt,struct output_ctx *octx);json_t *optstrip_stmt_json (const struct stmt *stmt,struct output_ctx *octx); +json_t *xt_stmt_json (const struct stmt *stmt,struct output_ctx *octx);int do_command_list_json (struct netlink_ctx *ctx,struct cmd *cmd);
@@ -194,6 +195,7 @@ STMT_PRINT_STUB(connlimit) STMT_PRINT_STUB(tproxy) STMT_PRINT_STUB(synproxy) STMT_PRINT_STUB(optstrip) +STMT_PRINT_STUB(xt)#undef STMT_PRINT_STUB#undef EXPR_PRINT_STUB
include/parser.h +1 -0
@@ -83,6 +83,7 @@ enum startcond_type {PARSER_SC_STMT_REJECT ,PARSER_SC_STMT_SYNPROXY ,PARSER_SC_STMT_TPROXY , +PARSER_SC_STMT_XT , __SC_MAX };
include/xt.h +4 -0
@@ -26,4 +26,8 @@ static inline void stmt_xt_postprocess(struct rule_pp_ctx *rctx,#endif +unsigned char *xt_stmt_blob_encode (const struct stmt *stmt); +int xt_stmt_blob_decode (struct stmt *stmt,const char *b64_string, +const struct location *b64_loc,struct list_head *msgs); +#endif /* _NFT_XT_H_ */
src/Makefile.am +2 -1
@@ -36,6 +36,7 @@ BUILT_SOURCES = parser_bison.h lib_LTLIBRARIES = libnftables.la libnftables_la_SOURCES = \ + base64.c \ rule.c \ statement.c \ cache.c \
@@ -89,7 +90,7 @@ libparser_la_CFLAGS = ${AM_CFLAGS} \ -Wno-undef \ -Wno-redundant-decls -libnftables_la_LIBADD = ${LIBMNL_LIBS} ${LIBNFTNL_LIBS} libparser.la +libnftables_la_LIBADD = ${LIBMNL_LIBS} ${LIBNFTNL_LIBS} ${Z_LIBS} libparser.la libnftables_la_LDFLAGS = -version-info ${libnftables_LIBVERSION} \ -Wl,--version-script=$(srcdir)/libnftables.map
src/base64.c new file +170 -0
170 lines not on this page · raw message
src/evaluate.c +1 -0
@@ -3989,6 +3989,7 @@ int stmt_evaluate(struct eval_ctx *ctx, struct stmt *stmt)case STMT_QUOTA :case STMT_NOTRACK :case STMT_FLOW_OFFLOAD : +case STMT_XT :return 0 ;case STMT_EXPRESSION :return stmt_evaluate_expr (ctx, stmt);
src/json.c +19 -6
@@ -7,6 +7,7 @@#include <netlink.h> #include <rule.h> #include <rt.h> +#include <xt.h> #include <netdb.h> #include <netinet/icmp6.h>
@@ -82,12 +83,6 @@ static json_t *stmt_print_json(const struct stmt *stmt, struct output_ctx *octx)char buf[1024 ];FILE *fp; -/* XXX: Can't be supported at this point: -* xt_stmt_xlate() ignores output_fp. -*/ -if (stmt->ops->type ==STMT_XT ) -return json_pack ("{s:n}" ,"xt" ); -if (stmt->ops->json)return stmt->ops->json (stmt, octx);
@@ -1624,6 +1619,24 @@ json_t *optstrip_stmt_json(const struct stmt *stmt, struct output_ctx *octx) expr_print_json(stmt->optstrip.expr, octx)); } +json_t *xt_stmt_json (const struct stmt *stmt,struct output_ctx *octx) +{ +static const char *xt_typename[] = { + [NFT_XT_MATCH ] ="match" , + [NFT_XT_TARGET ] ="target" , + [NFT_XT_WATCHER ] ="watcher" , + [NFT_XT_MAX ] ="unknown" + }; +unsigned char *b64_buf; +json_t *json; + + b64_buf =xt_stmt_blob_encode (stmt); + json =json_pack ("[s, s, s]" , xt_typename[stmt->xt.type], + stmt->xt.name, b64_buf); +xfree (b64_buf); +return json; +} +static json_t *table_print_json_full (struct netlink_ctx *ctx,struct table *table) {
src/netlink_linearize.c +32 -0
@@ -20,6 +20,7 @@#include <gmputil.h> #include <utils.h> #include <netinet/in.h> +#include <zlib.h> #include <linux/netfilter.h> #include <libnftnl/udata.h>
@@ -1365,6 +1366,35 @@ static void netlink_gen_optstrip_stmt(struct netlink_linearize_ctx *ctx,nft_rule_add_expr (ctx, nle, &expr->location); } +static void netlink_gen_xt_stmt (struct netlink_linearize_ctx *ctx, +const struct stmt *stmt) +{ +void *data =xmalloc (stmt->xt.infolen); +struct nftnl_expr *nle; + +memcpy (data, stmt->xt.info, stmt->xt.infolen); + +switch (stmt->xt.type) { +case NFT_XT_MATCH : + nle =alloc_nft_expr ("match" ); +nftnl_expr_set_str (nle,NFTNL_EXPR_MT_NAME , stmt->xt.name); +nftnl_expr_set_u32 (nle,NFTNL_EXPR_MT_REV , stmt->xt.rev); +nftnl_expr_set (nle,NFTNL_EXPR_MT_INFO , data, stmt->xt.infolen); +break ; +case NFT_XT_TARGET : +case NFT_XT_WATCHER : + nle =alloc_nft_expr ("target" ); +nftnl_expr_set_str (nle,NFTNL_EXPR_TG_NAME , stmt->xt.name); +nftnl_expr_set_u32 (nle,NFTNL_EXPR_TG_REV , stmt->xt.rev); +nftnl_expr_set (nle,NFTNL_EXPR_TG_INFO , data, stmt->xt.infolen); +break ; +default : +free (data); +return ; + } +nft_rule_add_expr (ctx, nle, &stmt->location); +} +static void netlink_gen_queue_stmt (struct netlink_linearize_ctx *ctx,const struct stmt *stmt) {
@@ -1630,6 +1660,8 @@ static void netlink_gen_stmt(struct netlink_linearize_ctx *ctx,return netlink_gen_chain_stmt (ctx, stmt);case STMT_OPTSTRIP :return netlink_gen_optstrip_stmt (ctx, stmt); +case STMT_XT : +return netlink_gen_xt_stmt (ctx, stmt);default :BUG ("unknown statement type %s\n" , stmt->ops->name); }
src/parser_bison.y +28 -0
@@ -626,6 +626,13 @@ int nft_lex(void *, void *, void *); %token IN "in" %token OUT "out" +%token XT "xt" +%token MATCH "match" +%token TARGET "target" +%token WATCHER "watcher" +%type <stmt> xt_stmt +%type <val> xt_stmt_type + %type <limit_rate> limit_rate_pkts %type <limit_rate> limit_rate_bytes
@@ -991,6 +998,7 @@ close_scope_udplite : { scanner_pop_start_cond(nft->scanner, PARSER_SC_EXPR_UDPL close_scope_log : { scanner_pop_start_cond(nft->scanner, PARSER_SC_STMT_LOG); } close_scope_synproxy : { scanner_pop_start_cond(nft->scanner, PARSER_SC_STMT_SYNPROXY); } +close_scope_xt : { scanner_pop_start_cond(nft->scanner, PARSER_SC_STMT_XT); } common_block : INCLUDE QUOTED_STRING stmt_separator {
@@ -2879,6 +2887,26 @@ stmt : verdict_stmt | synproxy_stmt close_scope_synproxy | chain_stmt | optstrip_stmt + | xt_stmt + ; + +xt_stmt_type : MATCH { $$ = NFT_XT_MATCH; } + | TARGET { $$ = NFT_XT_TARGET; } + | WATCHER { $$ = NFT_XT_WATCHER; } + ; + +xt_stmt : XT xt_stmt_type STRING STRING close_scope_xt + { + $$ = xt_stmt_alloc(&@$); + $$->xt.type = $2; + $$->xt.name = $3; + if (xt_stmt_blob_decode($$, $4, &@4, state->msgs)) { + xfree($$); + xfree($4); + YYERROR; + } + xfree($4); + } ; chain_stmt_type : JUMP { $$ = NFT_JUMP; }
src/parser_json.c +36 -0
@@ -13,6 +13,7 @@#include <rule.h> #include <sctp_chunk.h> #include <socket.h> +#include <xt.h> #include <netdb.h> #include <netinet/icmp6.h>
@@ -2707,6 +2708,40 @@ static struct stmt *json_parse_optstrip_stmt(struct json_ctx *ctx,return expr ?optstrip_stmt_alloc (int_loc, expr) :NULL ; } +static enum nft_xt_type xt_stmt_typeval (const char *type) +{ +if (!strcmp (type,"match" )) +return NFT_XT_MATCH ; +if (!strcmp (type,"target" )) +return NFT_XT_TARGET ; +if (!strcmp (type,"watcher" )) +return NFT_XT_WATCHER ; +return NFT_XT_MAX ; +} + +static struct stmt *json_parse_xt_stmt (struct json_ctx *ctx, +const char *key,json_t *value) +{ +char *type, *name, *blob; +struct stmt *stmt; + +if (json_unpack_err (ctx, value,"[s, s, s]" , &type, &name, &blob)) +return NULL ; + + stmt =xt_stmt_alloc (int_loc); + stmt->xt.type =xt_stmt_typeval (type); +if (stmt->xt.type ==NFT_XT_MAX ) { +json_error (ctx,"Invalid xt type name '%s'." , type); +return NULL ; + } + stmt->xt.name =xstrdup (name); +if (xt_stmt_blob_decode (stmt, blob, int_loc, ctx->msgs)) { +json_error (ctx,"Invalid base64 data in '%s'." , blob); +return NULL ; + } +return stmt; +} +static struct stmt *json_parse_stmt (struct json_ctx *ctx,json_t *root) {struct {
@@ -2745,6 +2780,7 @@ static struct stmt *json_parse_stmt(struct json_ctx *ctx, json_t *root) { "synproxy", json_parse_synproxy_stmt }, { "reset", json_parse_optstrip_stmt }, { "secmark", json_parse_secmark_stmt }, + { "xt", json_parse_xt_stmt }, };const char *type;unsigned int i;
src/scanner.l +14 -0
@@ -125,6 +125,7 @@ quotedstring \"[^"]*\" asteriskstring ({string}\*|{string}\\\*|\\\*|{string}\\\*{string}) comment #.*$ slash \/ +base64string ({letter}|{digit}|\+|{slash})*=* timestring ([0-9]+d)?([0-9]+h)?([0-9]+m)?([0-9]+s)?([0-9]+ms)?
@@ -247,6 +248,7 @@ addrstring ({macaddr}|{ip4addr}|{ip6addr}) %s SCANSTATE_STMT_REJECT %s SCANSTATE_STMT_SYNPROXY %s SCANSTATE_STMT_TPROXY +%s SCANSTATE_STMT_XT %%
@@ -411,6 +413,13 @@ addrstring ({macaddr}|{ip4addr}|{ip6addr}) "group" { return GROUP; } } +"xt" { scanner_push_start_cond(yyscanner, SCANSTATE_STMT_XT); return XT; } +<SCANSTATE_STMT_XT>{ + "match" { return MATCH; } + "target" { return TARGET; } + "watcher" { return WATCHER; } +} + "queue" { scanner_push_start_cond(yyscanner, SCANSTATE_EXPR_QUEUE); return QUEUE;} <SCANSTATE_EXPR_QUEUE>{ "num" { return QUEUENUM;}
@@ -846,6 +855,11 @@ addrstring ({macaddr}|{ip4addr}|{ip6addr}) return STRING; } +{base64string} { + yylval->string = xstrdup(yytext); + return STRING; + } + \\{newline} { reset_pos(yyget_extra(yyscanner), yylloc); }
src/statement.c +1 -0
@@ -997,6 +997,7 @@ static const struct stmt_ops xt_stmt_ops = { .name = "xt", .print = xt_stmt_print, .destroy = xt_stmt_destroy, + .json = xt_stmt_json, };struct stmt *xt_stmt_alloc (const struct location *loc)
src/xt.c +91 -8
133 lines not on this page · raw message
Signature · 1 line
2.38.0
Phil Sutter
reviewerPATCH v2 3/4 xt: Put match/target translation into own functions
In reply to Phil SutterReduce the size and indenting level of xt_stmt_xlate() a bit, also fix for error printing to stderr irrespective of octx->error_fp value.
---
Changes since v1:
- Change fprintf() calls to respect octx.
--- src/xt.c | 144 ++++++++++++++++++++++++++++++------------------------- 1 file changed, 78 insertions(+), 66 deletions(-)
src/xt.c +78 -66
@@ -104,6 +104,75 @@ int xt_stmt_blob_decode(struct stmt *stmt, const char *b64_string,return ret; } +#ifdef HAVE_LIBXTABLES +static bool xt_stmt_xlate_match (const struct stmt *stmt,void *entry, +struct xt_xlate *xl,struct output_ctx *octx) +{ +size_t size =XT_ALIGN (sizeof (struct xt_entry_match )) + + stmt->xt.infolen; +struct xt_xlate_mt_params params = { + .ip = entry, + .numeric =1 , + }; +struct xtables_match *mt; +struct xt_entry_match *m; + + mt =xtables_find_match (stmt->xt.name,XTF_TRY_LOAD ,NULL ); +if (!mt) { +fprintf (octx->error_fp, +"XT match %s not found\n" , stmt->xt.name); +return false; + } +if (!mt->xlate) +return false; + + m =xzalloc (size); + m->u.match_size = size; + m->u.user.revision = stmt->xt.rev; +memcpy (&m->data, stmt->xt.info, stmt->xt.infolen); + + params.match = m; + mt->xlate (xl, ¶ms); + +xfree (m); +return true; +} + +static bool xt_stmt_xlate_target (const struct stmt *stmt,void *entry, +struct xt_xlate *xl,struct output_ctx *octx) +{ +size_t size =XT_ALIGN (sizeof (struct xt_entry_target )) + + stmt->xt.infolen; +struct xt_xlate_tg_params params = { + .ip = entry, + .numeric =1 , + }; +struct xtables_target *tg; +struct xt_entry_target *t; + + tg =xtables_find_target (stmt->xt.name,XTF_TRY_LOAD ); +if (!tg) { +fprintf (octx->error_fp, +"XT target %s not found\n" , stmt->xt.name); +return false; + } +if (!tg->xlate) +return false; + + t =xzalloc (size); + t->u.target_size = size; + t->u.user.revision = stmt->xt.rev; +memcpy (&t->data, stmt->xt.info, stmt->xt.infolen); +strcpy (t->u.user.name, tg->name); + + params.target = t; + tg->xlate (xl, ¶ms); + +xfree (t); +return true; +} +#endif +void xt_stmt_xlate (const struct stmt *stmt,struct output_ctx *octx) {static const char *xt_typename[] = {
@@ -115,11 +184,7 @@ void xt_stmt_xlate(const struct stmt *stmt, struct output_ctx *octx)unsigned char *b64_buf;#ifdef HAVE_LIBXTABLES struct xt_xlate *xl =xt_xlate_alloc (10240 ); -struct xtables_target *tg; -struct xt_entry_target *t; -struct xtables_match *mt; -struct xt_entry_match *m; -size_t size; +bool xlated = false;void *entry;xtables_set_nfproto (stmt->xt.family);
@@ -127,76 +192,23 @@ void xt_stmt_xlate(const struct stmt *stmt, struct output_ctx *octx)switch (stmt->xt.type) {case NFT_XT_MATCH : - mt =xtables_find_match (stmt->xt.name,XTF_TRY_LOAD ,NULL ); -if (!mt) { -fprintf (stderr,"XT match %s not found\n" , - stmt->xt.name); -return ; - } - size =XT_ALIGN (sizeof (*m)) + stmt->xt.infolen; - - m =xzalloc (size); -memcpy (&m->data, stmt->xt.info, stmt->xt.infolen); - - m->u.match_size = size; - m->u.user.revision = stmt->xt.rev; - -if (mt->xlate) { -struct xt_xlate_mt_params params = { - .ip = entry, - .match = m, - .numeric =1 , - }; - - mt->xlate (xl, ¶ms); -nft_print (octx,"%s" ,xt_xlate_get (xl)); -xfree (m); -xfree (entry); -xt_xlate_free (xl); -return ; - } -xfree (m); + xlated =xt_stmt_xlate_match (stmt, entry, xl, octx);break ;case NFT_XT_WATCHER :case NFT_XT_TARGET : - tg =xtables_find_target (stmt->xt.name,XTF_TRY_LOAD ); -if (!tg) { -fprintf (stderr,"XT target %s not found\n" , - stmt->xt.name); -return ; - } - size =XT_ALIGN (sizeof (*t)) + stmt->xt.infolen; - - t =xzalloc (size); -memcpy (&t->data, stmt->xt.info, stmt->xt.infolen); - - t->u.target_size = size; - t->u.user.revision = stmt->xt.rev; - -strcpy (t->u.user.name, tg->name); - -if (tg->xlate) { -struct xt_xlate_tg_params params = { - .ip = entry, - .target = t, - .numeric =1 , - }; - - tg->xlate (xl, ¶ms); -nft_print (octx,"%s" ,xt_xlate_get (xl)); -xfree (t); -xfree (entry); -xt_xlate_free (xl); -return ; - } -xfree (t); + xlated =xt_stmt_xlate_target (stmt, entry, xl, octx);break ;default :break ; } -xt_xlate_free (xl);xfree (entry); +if (xlated) { +nft_print (octx,"%s" ,xt_xlate_get (xl)); +xt_xlate_free (xl); +return ; + } +xt_xlate_free (xl);#endif b64_buf =xt_stmt_blob_encode (stmt); nft_print(octx, "xt %s %s %s",
Signature · 1 line
2.38.0
Phil Sutter
reviewerPATCH v2 4/4 xt: Detect xlate callback failure
In reply to Phil SutterIf an extension's xlate callback returns 0, translation is at least incomplete. Discard the result and resort to opaque dump format in this case.
--- src/xt.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
src/xt.c +6 -4
@@ -116,6 +116,7 @@ static bool xt_stmt_xlate_match(const struct stmt *stmt, void *entry, };struct xtables_match *mt;struct xt_entry_match *m; +int rc; mt =xtables_find_match (stmt->xt.name,XTF_TRY_LOAD ,NULL );if (!mt) {
@@ -132,10 +133,10 @@ static bool xt_stmt_xlate_match(const struct stmt *stmt, void *entry,memcpy (&m->data, stmt->xt.info, stmt->xt.infolen); params.match = m; - mt->xlate (xl, ¶ms); + rc = mt->xlate (xl, ¶ms);xfree (m); -return true; +return rc !=0 ; } static bool xt_stmt_xlate_target(const struct stmt *stmt, void *entry,
@@ -149,6 +150,7 @@ static bool xt_stmt_xlate_target(const struct stmt *stmt, void *entry, };struct xtables_target *tg;struct xt_entry_target *t; +int rc; tg =xtables_find_target (stmt->xt.name,XTF_TRY_LOAD );if (!tg) {
@@ -166,10 +168,10 @@ static bool xt_stmt_xlate_target(const struct stmt *stmt, void *entry,strcpy (t->u.user.name, tg->name); params.target = t; - tg->xlate (xl, ¶ms); + rc = tg->xlate (xl, ¶ms);xfree (t); - return true; + return rc != 0; }#endif
Signature · 1 line
2.38.0
Florian Westphal
maintainerQuoted 3 lines · Phil Sutter
If nft can't translate a compat expression, dump it in a format that can be restored later without losing data, thereby keeping the ruleset intact.
Why? :-( This cements nft_compat.c forever.
If we're goping to do it lets at least dump it properly, i.e. nft ... add rule compat "-m conntrack --ctstate NEW".
At this time I'd rather like a time machine to prevent nft_compat.c from getting merged :-(
Pablo Neira Ayuso
maintainerQuoted 9 lines · Florian Westphal
Quoted 3 lines · Phil Sutter
If nft can't translate a compat expression, dump it in a format that can be restored later without losing data, thereby keeping the ruleset intact.
Why? :-( This cements
nft_compat.cforever.If we're goping to do it lets at least dump it properly, i.e. nft ... add rule compat "-m conntrack --ctstate NEW".
We might have to support this mixed syntax forever. I proposed this long long time ago, when nftables was supporting ~25% of the iptables feature-set. These days, where they are almost on par (actually nftables being a lot more expressive than iptables), I am not sure it makes sense to follow this path anymore.
Unless you refer to dumping a listing which nft cannot load as a way to provide a listing that is comprehensible, but that cannot be loaded by the user.
Quoted 2 lines
At this time I'd rather like a time machine to prevent
nft_compat.cfrom getting merged :-(
I agree we should have added native translations for iptables-nft sooner, but it was never a priority for anyone so far.
This "forward compatibility" issue (pretending old tool versions can interpret new revisions / features loaded by newer tool versions) we are trying to deal is hard, we already discussed none of the other existing tooling (ethtool, iproute2, etc.) supports for this.
If you prefer to go for the _USERDATA area as a last resort, I'm OK with it, this requires no kernel patches, and it will be used only for the "forward compatibility" scenario (last resort)
We can also resort on displaying the raw expressions, so the user gets a meaningful output that cannot be loaded again.
I think this more or less a summary of what we discussed in the NFWS.
Phil Sutter
reviewerQuoted 17 lines · Pablo Neira Ayuso
Quoted 9 lines · Florian Westphal
Quoted 3 lines · Phil Sutter
If nft can't translate a compat expression, dump it in a format that can be restored later without losing data, thereby keeping the ruleset intact.
Why? :-( This cements
nft_compat.cforever.If we're goping to do it lets at least dump it properly, i.e. nft ... add rule compat "-m conntrack --ctstate NEW".
We might have to support this mixed syntax forever. I proposed this long long time ago, when nftables was supporting ~25% of the iptables feature-set. These days, where they are almost on par (actually nftables being a lot more expressive than iptables), I am not sure it makes sense to follow this path anymore.
Quoted 3 lines
Unless you refer to dumping a listing which nft cannot load as a way to provide a listing that is comprehensible, but that cannot be loaded by the user.
Or a big fat warning, but I fear anything that allows | nft list ruleset | nft -f -
will simply be ignored until the problems become obvious.
Quoted 11 lines
Quoted 2 lines
At this time I'd rather like a time machine to prevent
nft_compat.cfrom getting merged :-(I agree we should have added native translations for iptables-nft sooner, but it was never a priority for anyone so far.
This "forward compatibility" issue (pretending old tool versions can interpret new revisions / features loaded by newer tool versions) we are trying to deal is hard, we already discussed none of the other existing tooling (ethtool, iproute2, etc.) supports for this.
Quoted 8 lines
If you prefer to go for the
_USERDATAarea as a last resort, I'm OK with it, this requires no kernel patches, and it will be used only for the "forward compatibility" scenario (last resort)We can also resort on displaying the raw expressions, so the user gets a meaningful output that cannot be loaded again.
I think this more or less a summary of what we discussed in the NFWS.
Pablo, I think you're mixing up two things here:
This "support dump and load of compat expression" feature is to sanitize the current situation with up to date iptables and nftables.
The forward compat (you were right about the direction) issue is mostly about iptables-nft vs. itself in an older version.
Cheers, Phil
Phil Sutter
reviewerQuoted 6 lines · Florian Westphal
Quoted 3 lines · Phil Sutter
If nft can't translate a compat expression, dump it in a format that can be restored later without losing data, thereby keeping the ruleset intact.
Why? :-( This cements
nft_compat.cforever.
To avoid silently breaking a ruleset. It is a last resort measure for cases where nft can't translate the ruleset into something meaningful. I already submitted a patch adding a warning when listing tables containing compat expressions (status unclear), but a real alternative to the above would be to abort the listing or to ignore the table.
Listing the ruleset in translated form when iptables-nft can't parse the translation or without translation but still maintaining syntax to be parsed without error during a later restore is almost luring users into doing stupid things.
Quoted 2 lines
If we're goping to do it lets at least dump it properly, i.e. nft ... add rule compat "-m conntrack --ctstate NEW".
This will make things worse: People will understand the format and start using it despite the warnings. This adds a new user base to compat expressions. I don't dare claiming nobody would start crafting compat expressions in my zip-dump format, but it's at least a larger obstacle. :D
Quoted 2 lines
At this time I'd rather like a time machine to prevent
nft_compat.cfrom getting merged :-(
If you do, please convince Pablo to not push iptables commit 384958620a. I think it opened the can of worms we're trying to confine here.
Cheers, Phil
Pablo Neira Ayuso
maintainerMerging threads.
On Fri, Nov 18, 2022 at 10:55:04AM +0100, Phil Sutter wrote:
[...]
Quoted 6 lines
Quoted 1 line
I think this more or less a summary of what we discussed in the NFWS.
Pablo, I think you're mixing up two things here:
This "support dump and load of compat expression" feature is to sanitize the current situation with up to date iptables and nftables.
OK, then the problem we discuss is mixing iptables-nft and nftables.
On Fri, Nov 18, 2022 at 10:47:48AM +0100, Phil Sutter wrote:
[...]
Quoted 5 lines
Quoted 2 lines
At this time I'd rather like a time machine to prevent
nft_compat.cfrom getting merged :-(If you do, please convince Pablo to not push iptables commit
384958620a. I think it opened the can of worms we're trying to confine here.
It could be worst, if iptables-nft would not be in place, then old iptables-legacy and new nftables rules would have no visibility each other.
With iptables-nft we have a way to move forward:
- Replace nft_compat by native expressions from iptables-nft.
- Extend iptables-nft to understand more complex expressions, worst case dump a native representation.
Why don't we just move ahead this path instead of spinning around the compat layer? This only requires userspace updates on iptables-nft.
Phil Sutter
reviewerQuoted 25 lines · Pablo Neira Ayuso
Merging threads.
On Fri, Nov 18, 2022 at 10:55:04AM +0100, Phil Sutter wrote:
[...]Quoted 6 lines
Quoted 1 line
I think this more or less a summary of what we discussed in the NFWS.
Pablo, I think you're mixing up two things here:
This "support dump and load of compat expression" feature is to sanitize the current situation with up to date iptables and nftables.
OK, then the problem we discuss is mixing iptables-nft and nftables.
On Fri, Nov 18, 2022 at 10:47:48AM +0100, Phil Sutter wrote:
[...]Quoted 5 lines
Quoted 2 lines
At this time I'd rather like a time machine to prevent
nft_compat.cfrom getting merged :-(If you do, please convince Pablo to not push iptables commit
384958620a. I think it opened the can of worms we're trying to confine here.It could be worst, if iptables-nft would not be in place, then old iptables-legacy and new nftables rules would have no visibility each other.
Quoted 8 lines
With iptables-nft we have a way to move forward:
- Replace
nft_compatby native expressions from iptables-nft.
- Extend iptables-nft to understand more complex expressions, worst case dump a native representation.Why don't we just move ahead this path instead of spinning around the compat layer? This only requires userspace updates on iptables-nft.
Sure! I'm just picking low hanging fruits first. With even translation support being still incomplete, I fear it will take a while until the tools are fluent enough for this to not matter anymore. And then there's still nftables without libxtables support.
Cheers, Phil
Florian Westphal
maintainerQuoted 32 lines · Phil Sutter
Quoted 31 lines · Pablo Neira Ayuso
Merging threads.
On Fri, Nov 18, 2022 at 10:55:04AM +0100, Phil Sutter wrote:
[...]Quoted 6 lines
Quoted 1 line
I think this more or less a summary of what we discussed in the NFWS.
Pablo, I think you're mixing up two things here:
This "support dump and load of compat expression" feature is to sanitize the current situation with up to date iptables and nftables.
OK, then the problem we discuss is mixing iptables-nft and nftables.
On Fri, Nov 18, 2022 at 10:47:48AM +0100, Phil Sutter wrote:
[...]Quoted 5 lines
Quoted 2 lines
At this time I'd rather like a time machine to prevent
nft_compat.cfrom getting merged :-(If you do, please convince Pablo to not push iptables commit
384958620a. I think it opened the can of worms we're trying to confine here.It could be worst, if iptables-nft would not be in place, then old iptables-legacy and new nftables rules would have no visibility each other.
With iptables-nft we have a way to move forward:
- Replace
nft_compatby native expressions from iptables-nft.
- Extend iptables-nft to understand more complex expressions, worst case dump a native representation.
Quoted 7 lines
Quoted 2 lines
Why don't we just move ahead this path instead of spinning around the compat layer? This only requires userspace updates on iptables-nft.
Sure! I'm just picking low hanging fruits first. With even translation support being still incomplete, I fear it will take a while until the tools are fluent enough for this to not matter anymore. And then there's still nftables without libxtables support.
Then perhaps its better to do following path:
1. Try ->xlate(), if that fails, then print a 'breaking' format?
As far as I understand the problem is the "# comment" - type syntax that makes nft just skip the incomplete rule, so perhaps just use invalid format?
Example:
counter packets 0 bytes 0 # name foo interval 250.0ms ewmalog 500.0ms
Instead make this something like
counter packets 0 bytes 0 nft_compat [ RATEEST name foo interval 250.0ms ewmalog 500.0ms ] # unsupported iptables-nft rule
?
I'd like to avoid exposure in the frontend with compatible-restore-approach if possible.
Phil Sutter
reviewerQuoted 50 lines · Florian Westphal
Quoted 32 lines · Phil Sutter
Quoted 31 lines · Pablo Neira Ayuso
Merging threads.
On Fri, Nov 18, 2022 at 10:55:04AM +0100, Phil Sutter wrote:
[...]Quoted 6 lines
Quoted 1 line
I think this more or less a summary of what we discussed in the NFWS.
Pablo, I think you're mixing up two things here:
This "support dump and load of compat expression" feature is to sanitize the current situation with up to date iptables and nftables.
OK, then the problem we discuss is mixing iptables-nft and nftables.
On Fri, Nov 18, 2022 at 10:47:48AM +0100, Phil Sutter wrote:
[...]Quoted 5 lines
Quoted 2 lines
At this time I'd rather like a time machine to prevent
nft_compat.cfrom getting merged :-(If you do, please convince Pablo to not push iptables commit
384958620a. I think it opened the can of worms we're trying to confine here.It could be worst, if iptables-nft would not be in place, then old iptables-legacy and new nftables rules would have no visibility each other.
With iptables-nft we have a way to move forward:
- Replace
nft_compatby native expressions from iptables-nft.
- Extend iptables-nft to understand more complex expressions, worst case dump a native representation.Quoted 7 lines
Quoted 2 lines
Why don't we just move ahead this path instead of spinning around the compat layer? This only requires userspace updates on iptables-nft.
Sure! I'm just picking low hanging fruits first. With even translation support being still incomplete, I fear it will take a while until the tools are fluent enough for this to not matter anymore. And then there's still nftables without libxtables support.
Then perhaps its better to do following path:
1. Try ->xlate(), if that fails, then print a 'breaking' format?As far as I understand the problem is the "# comment" - type syntax that makes nft just skip the incomplete rule, so perhaps just use invalid format?
Example:
Quoted 7 lines
counter packets 0 bytes 0 # name foo interval 250.0ms ewmalog 500.0ms
Instead make this something like
counter packets 0 bytes 0nft_compat[ RATEEST name foo interval 250.0ms ewmalog 500.0ms ] # unsupported iptables-nft rule?
I'd like to avoid exposure in the frontend with compatible-restore-approach if possible.
Yes, that's fine with me. Now what about translated expressions? Can we apply my warning patch until at least the majority of them is understood by iptables?
Cheers, Phil
Pablo Neira Ayuso
maintainerQuoted 55 lines · Phil Sutter
Quoted 54 lines · Florian Westphal
Quoted 32 lines · Phil Sutter
Quoted 31 lines · Pablo Neira Ayuso
Merging threads.
On Fri, Nov 18, 2022 at 10:55:04AM +0100, Phil Sutter wrote:
[...]Quoted 6 lines
Quoted 1 line
I think this more or less a summary of what we discussed in the NFWS.
Pablo, I think you're mixing up two things here:
This "support dump and load of compat expression" feature is to sanitize the current situation with up to date iptables and nftables.
OK, then the problem we discuss is mixing iptables-nft and nftables.
On Fri, Nov 18, 2022 at 10:47:48AM +0100, Phil Sutter wrote:
[...]Quoted 5 lines
Quoted 2 lines
At this time I'd rather like a time machine to prevent
nft_compat.cfrom getting merged :-(If you do, please convince Pablo to not push iptables commit
384958620a. I think it opened the can of worms we're trying to confine here.It could be worst, if iptables-nft would not be in place, then old iptables-legacy and new nftables rules would have no visibility each other.
With iptables-nft we have a way to move forward:
- Replace
nft_compatby native expressions from iptables-nft.
- Extend iptables-nft to understand more complex expressions, worst case dump a native representation.Quoted 7 lines
Quoted 2 lines
Why don't we just move ahead this path instead of spinning around the compat layer? This only requires userspace updates on iptables-nft.
Sure! I'm just picking low hanging fruits first. With even translation support being still incomplete, I fear it will take a while until the tools are fluent enough for this to not matter anymore. And then there's still nftables without libxtables support.
Then perhaps its better to do following path:
1. Try ->xlate(), if that fails, then print a 'breaking' format?As far as I understand the problem is the "# comment" - type syntax that makes nft just skip the incomplete rule, so perhaps just use invalid format?
Example:
counter packets 0 bytes 0 # name foo interval 250.0ms ewmalog 500.0ms
Instead make this something like
counter packets 0 bytes 0nft_compat[ RATEEST name foo interval 250.0ms ewmalog 500.0ms ] # unsupported iptables-nft rule
Quoted 7 lines
Quoted 3 lines
?
I'd like to avoid exposure in the frontend with compatible-restore-approach if possible.
Yes, that's fine with me. Now what about translated expressions? Can we apply my warning patch until at least the majority of them is understood by iptables?
Which one are you refering to?
Phil Sutter
reviewerQuoted 58 lines · Pablo Neira Ayuso
Quoted 57 lines · Phil Sutter
Quoted 56 lines · Florian Westphal
Quoted 32 lines · Phil Sutter
Quoted 31 lines · Pablo Neira Ayuso
Merging threads.
On Fri, Nov 18, 2022 at 10:55:04AM +0100, Phil Sutter wrote:
[...]Quoted 6 lines
Quoted 1 line
I think this more or less a summary of what we discussed in the NFWS.
Pablo, I think you're mixing up two things here:
This "support dump and load of compat expression" feature is to sanitize the current situation with up to date iptables and nftables.
OK, then the problem we discuss is mixing iptables-nft and nftables.
On Fri, Nov 18, 2022 at 10:47:48AM +0100, Phil Sutter wrote:
[...]Quoted 5 lines
Quoted 2 lines
At this time I'd rather like a time machine to prevent
nft_compat.cfrom getting merged :-(If you do, please convince Pablo to not push iptables commit
384958620a. I think it opened the can of worms we're trying to confine here.It could be worst, if iptables-nft would not be in place, then old iptables-legacy and new nftables rules would have no visibility each other.
With iptables-nft we have a way to move forward:
- Replace
nft_compatby native expressions from iptables-nft.
- Extend iptables-nft to understand more complex expressions, worst case dump a native representation.Quoted 7 lines
Quoted 2 lines
Why don't we just move ahead this path instead of spinning around the compat layer? This only requires userspace updates on iptables-nft.
Sure! I'm just picking low hanging fruits first. With even translation support being still incomplete, I fear it will take a while until the tools are fluent enough for this to not matter anymore. And then there's still nftables without libxtables support.
Then perhaps its better to do following path:
1. Try ->xlate(), if that fails, then print a 'breaking' format?As far as I understand the problem is the "# comment" - type syntax that makes nft just skip the incomplete rule, so perhaps just use invalid format?
Example:
counter packets 0 bytes 0 # name foo interval 250.0ms ewmalog 500.0ms
Instead make this something like
counter packets 0 bytes 0nft_compat[ RATEEST name foo interval 250.0ms ewmalog 500.0ms ] # unsupported iptables-nft rule?
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
I'd like to avoid exposure in the frontend with compatible-restore-approach if possible.
Yes, that's fine with me. Now what about translated expressions? Can we apply my warning patch until at least the majority of them is understood by iptables?
Which one are you refering to?
This one:
Subject: [nft PATCH] Warn for tables with compat expressions in rules
Date: Wed, 12 Oct 2022 17:31:07 +0200
Message-Id: <[email protected]>
Florian Westphal
maintainerQuoted 5 lines · Phil Sutter
This one:
Subject: [nft PATCH] Warn for tables with compat expressions in rules
Date: Wed, 12 Oct 2022 17:31:07 +0200
Message-Id: <[email protected]>
This is:
https://patchwork.ozlabs.org/project/netfilter-devel/patch/[email protected]/
LGTM, no objections from me.
Pablo Neira Ayuso
maintainerQuoted 11 lines · Florian Westphal
Quoted 5 lines · Phil Sutter
This one:
Subject: [nft PATCH] Warn for tables with compat expressions in rules
Date: Wed, 12 Oct 2022 17:31:07 +0200
Message-Id: <[email protected]>This is:
https://patchwork.ozlabs.org/project/netfilter-devel/patch/[email protected]/LGTM, no objections from me.
LGTM.
Quoted 8 lines · Pablo Neira Ayuso
Quoted 7 lines · Florian Westphal
Quoted 5 lines · Phil Sutter
This one:
Subject: [nft PATCH] Warn for tables with compat expressions in rules
Date: Wed, 12 Oct 2022 17:31:07 +0200
Message-Id: <[email protected]>
Quoted 6 lines
Quoted 4 lines
This is:
https://patchwork.ozlabs.org/project/netfilter-devel/patch/[email protected]/LGTM, no objections from me.
LGTM.
Applied, thanks!