Skip to contentSkip to conversation

add typeof keyword

PATCH Superseded Florian Westphal 3 participants 15 messages Aug 16, 2019 – Aug 26, 2019 netfilter

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Florian Westphal

author maintainer
#0 lore raw

cover add typeof keyword

This patch series adds the typeof keyword.

The only dependency is a small change to libnftnl to add two new UDATA_SET_TYPEOF enum values.

named set can be configured as follows:

set os {
type typeof(osf name)
elements = { "Linux", "Windows" }
}

or
nft add set ip filter allowed "{ type typeof(ip daddr) . typeof(tcp dport); }"

... which is the same as the "old" 'type ipv4_addr . inet_service".

The type is stored in the kernel via the udata set infrastructure, on listing -- if a udata type is present -- nft will validate that this type matches the set key length.

This initial submission doesn't include a documentation update because I'd like to get feedback on the chosen syntax first.

Florian Westphal (8):
src: libnftnl: run single-initcalls only once
src: libnftnl: split nft_ctx_new/free
src: store expr, not dtype to track data in sets
src: parser: add syntax to provide bitsize for non-spcific types
src: add "typeof" keyword
src: add "typeof" print support
src: netlink: remove assertion
tests: add typeof test cases

 include/datatype.h                                 |    1 
 include/netlink.h                                  |    1 
 include/nftables.h                                 |    3 
 include/rule.h                                     |    6 
 src/datatype.c                                     |    5 
 src/evaluate.c                                     |   58 ++++--
 src/expression.c                                   |    2 
 src/json.c                                         |    4 
 src/libnftables.c                                  |   48 +++--
 src/mnl.c                                          |   39 ++++
 src/monitor.c                                      |    2 
 src/netlink.c                                      |  176 ++++++++++++++++++---
 src/netlink_delinearize.c                          |   15 +
 src/parser_bison.y                                 |   26 ++-
 src/parser_json.c                                  |    8 
 src/rule.c                                         |   35 +++-
 src/scanner.l                                      |    1 
 src/segtree.c                                      |    8 
 tests/shell/testcases/maps/dumps/typeof_maps_0.nft |   16 +
 tests/shell/testcases/maps/typeof_maps_0           |   26 +++
 tests/shell/testcases/sets/dumps/typeof_sets_0.nft |   31 +++
 tests/shell/testcases/sets/typeof_sets_0           |   40 ++++
 22 files changed, 459 insertions(+), 92 deletions(-)
author maintainer
#1 lore raw

PATCH 1/8 src: libnftnl: run single-initcalls only once

---
 src/libnftables.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
src/libnftables.c +9 -5
index a693c0c69075..b169dd2f2afe 100644
@@ -90,11 +90,6 @@ static void nft_init(struct nft_ctx *ctx)
 	realm_table_rt_init(ctx);
 	devgroup_table_init(ctx);
 	ct_label_table_init(ctx);
-
-	gmp_init();
-#ifdef HAVE_LIBXTABLES
-	xt_init();
-#endif
 }
 
 static void nft_exit(struct nft_ctx *ctx)
@@ -142,8 +137,17 @@ static void nft_ctx_netlink_init(struct nft_ctx *ctx)
 EXPORT_SYMBOL(nft_ctx_new);
 struct nft_ctx *nft_ctx_new(uint32_t flags)
 {
+	static bool init_once;
 	struct nft_ctx *ctx;
 
+	if (!init_once) {
+		init_once = true;
+		gmp_init();
+#ifdef HAVE_LIBXTABLES
+		xt_init();
+#endif
+	}
+
 	ctx = xzalloc(sizeof(struct nft_ctx));
 	nft_init(ctx);
 
Signature · 1 line
2.21.0
author maintainer
#2 lore raw

PATCH 2/8 src: libnftnl: split nft_ctx_new/free

In reply to Florian Westphal

a followup patch doesn't need e.g. symbol files to be loaded, so split those functions to create a helper that can be used internally by nft to create a more minimal nft context.

---
 include/nftables.h |  3 +++
 src/libnftables.c  | 32 ++++++++++++++++++++++----------
 2 files changed, 25 insertions(+), 10 deletions(-)
include/nftables.h +3 -0
index ef737c839b2e..b7fbcf5726d4 100644
@@ -201,6 +201,9 @@ int nft_print(struct output_ctx *octx, const char *fmt, ...)
 int nft_gmp_print(struct output_ctx *octx, const char *fmt, ...)
 	__attribute__((format(printf, 2, 0)));
 
+struct nft_ctx *__nft_ctx_new(void);
+void __nft_ctx_free(struct nft_ctx *ctx);
+
 #define __NFT_OUTPUT_NOTSUPP	UINT_MAX
 
 #endif /* NFTABLES_NFTABLES_H */
src/libnftables.c +22 -10
index b169dd2f2afe..a1e2fd662a7a 100644
@@ -134,8 +134,7 @@ static void nft_ctx_netlink_init(struct nft_ctx *ctx)
 	ctx->nf_sock = nft_mnl_socket_open();
 }
 
-EXPORT_SYMBOL(nft_ctx_new);
-struct nft_ctx *nft_ctx_new(uint32_t flags)
+struct nft_ctx *__nft_ctx_new(void)
 {
 	static bool init_once;
 	struct nft_ctx *ctx;
@@ -149,18 +148,26 @@ struct nft_ctx *nft_ctx_new(uint32_t flags)
 	}
 
 	ctx = xzalloc(sizeof(struct nft_ctx));
-	nft_init(ctx);
 
 	ctx->state = xzalloc(sizeof(struct parser_state));
 	nft_ctx_add_include_path(ctx, DEFAULT_INCLUDE_PATH);
-	ctx->parser_max_errors	= 10;
 	init_list_head(&ctx->cache.list);
-	ctx->flags = flags;
 	ctx->output.output_fp = stdout;
 	ctx->output.error_fp = stderr;
 
-	if (flags == NFT_CTX_DEFAULT)
-		nft_ctx_netlink_init(ctx);
+	return ctx;
+}
+
+EXPORT_SYMBOL(nft_ctx_new);
+struct nft_ctx *nft_ctx_new(uint32_t flags)
+{
+	struct nft_ctx *ctx = __nft_ctx_new();
+
+	nft_init(ctx);
+
+	ctx->parser_max_errors	= 10;
+	ctx->flags = flags;
+	nft_ctx_netlink_init(ctx);
 
 	return ctx;
 }
@@ -281,20 +288,25 @@ const char *nft_ctx_get_error_buffer(struct nft_ctx *ctx)
 	return get_cookie_buffer(&ctx->output.error_cookie);
 }
 
-EXPORT_SYMBOL(nft_ctx_free);
-void nft_ctx_free(struct nft_ctx *ctx)
+void __nft_ctx_free(struct nft_ctx *ctx)
 {
 	if (ctx->nf_sock)
 		mnl_socket_close(ctx->nf_sock);
 
 	exit_cookie(&ctx->output.output_cookie);
 	exit_cookie(&ctx->output.error_cookie);
-	iface_cache_release();
 	cache_release(&ctx->cache);
 	nft_ctx_clear_include_paths(ctx);
 	xfree(ctx->state);
 	xfree(ctx);
+}
+
+EXPORT_SYMBOL(nft_ctx_free);
+void nft_ctx_free(struct nft_ctx *ctx)
+{
 	nft_exit(ctx);
+	__nft_ctx_free(ctx);
+	iface_cache_release();
 }
 
 EXPORT_SYMBOL(nft_ctx_set_output);
Signature · 1 line
2.21.0
author maintainer
#3 lore raw

PATCH 3/8 src: store expr, not dtype to track data in sets

In reply to Florian Westphal

This will be needed once we add support for the 'typeof' keyword to handle maps that could e.g. store 'ct helper' "type" values.

Instead of:

set foo {
	type ipv4_addr . mark;

this would allow

set foo {
	typeof(ip saddr) . typeof(ct mark);

(exact syntax TBD).

This would be needed to allow sets that store variable-sized data types (string, integer and the like) that can't be used at at the moment.

Adding special data types for everything is problematic due to the large amount of different types needed.

For anonymous sets, e.g. "string" can be used because the needed size can be inferred from the statement, e.g. 'osf name { "Windows", "Linux }', but in case of named sets that won't work because 'type string' lacks the context needed to derive the size information.

With 'typeof(osf name)' the context is there, but at the moment it won't help because the expression is discarded instantly and only the data type is retained.

---
 include/datatype.h |  1 -
 include/netlink.h  |  1 -
 include/rule.h     |  6 ++---
 src/datatype.c     |  5 ----
 src/evaluate.c     | 58 ++++++++++++++++++++++++++++++++--------------
 src/expression.c   |  2 +-
 src/json.c         |  4 ++--
 src/mnl.c          |  6 ++---
 src/monitor.c      |  2 +-
 src/netlink.c      | 32 ++++++++++++-------------
 src/parser_bison.y |  3 +--
 src/parser_json.c  |  8 +++++--
 src/rule.c         |  8 +++----
 src/segtree.c      |  8 +++++--
 14 files changed, 81 insertions(+), 63 deletions(-)
include/datatype.h +0 -1
index c1d08cc29e47..87fef59dd1a6 100644
@@ -287,7 +287,6 @@ concat_subtype_lookup(uint32_t type, unsigned int n)
 
 extern const struct datatype *
 set_datatype_alloc(const struct datatype *orig_dtype, unsigned int byteorder);
-extern void set_datatype_destroy(const struct datatype *dtype);
 
 extern void time_print(uint64_t msec, struct output_ctx *octx);
 extern struct error_record *time_parse(const struct location *loc,
include/netlink.h +0 -1
index 279723f33d31..6aead9a8c5e2 100644
@@ -190,6 +190,5 @@ int netlink_events_trace_cb(const struct nlmsghdr *nlh, int type,
 			    struct netlink_mon_handler *monh);
 
 enum nft_data_types dtype_map_to_kernel(const struct datatype *dtype);
-const struct datatype *dtype_map_from_kernel(enum nft_data_types type);
 
 #endif /* NFTABLES_NETLINK_H */
include/rule.h +2 -4
index 0ef6aacdd71b..331815643362 100644
@@ -274,8 +274,7 @@ extern struct rule *rule_lookup_by_index(const struct chain *chain,
  * @gc_int:	garbage collection interval
  * @timeout:	default timeout value
  * @key:	key expression (data type, length))
- * @datatype:	mapping data type
- * @datalen:	mapping data len
+ * @data:	mapping data expression
  * @objtype:	mapping object type
  * @init:	initializer
  * @rg_cache:	cached range element (left)
@@ -292,8 +291,7 @@ struct set {
 	uint32_t		gc_int;
 	uint64_t		timeout;
 	struct expr		*key;
-	const struct datatype	*datatype;
-	unsigned int		datalen;
+	struct expr		*data;
 	uint32_t		objtype;
 	struct expr		*init;
 	struct expr		*rg_cache;
src/datatype.c +0 -5
index c5a013463eb9..2111a72d4bed 100644
@@ -1187,11 +1187,6 @@ const struct datatype *set_datatype_alloc(const struct datatype *orig_dtype,
 	return dtype;
 }
 
-void set_datatype_destroy(const struct datatype *dtype)
-{
-	datatype_free(dtype);
-}
-
 static struct error_record *time_unit_parse(const struct location *loc,
 					    const char *str, uint64_t *unit)
 {
src/evaluate.c +40 -18

118 lines not on this page · raw message

src/expression.c +1 -1
index cb49e0b73f5a..09005c650ede 100644
@@ -994,7 +994,7 @@ static void map_expr_print(const struct expr *expr, struct output_ctx *octx)
 {
 	expr_print(expr->map, octx);
 	if (expr->mappings->etype == EXPR_SET_REF &&
-	    expr->mappings->set->datatype->type == TYPE_VERDICT)
+	    expr->mappings->set->data->dtype->type == TYPE_VERDICT)
 		nft_print(octx, " vmap ");
 	else
 		nft_print(octx, " map ");
src/json.c +2 -2
index 55ce05315d37..3481f2e0677f 100644
@@ -82,7 +82,7 @@ static json_t *set_print_json(struct output_ctx *octx, const struct set *set)
 
 	if (set_is_datamap(set->flags)) {
 		type = "map";
-		datatype_ext = set->datatype->name;
+		datatype_ext = set->data->dtype->name;
 	} else if (set_is_objmap(set->flags)) {
 		type = "map";
 		datatype_ext = obj_type_name(set->objtype);
@@ -617,7 +617,7 @@ json_t *map_expr_json(const struct expr *expr, struct output_ctx *octx)
 	const char *type = "map";
 
 	if (expr->mappings->etype == EXPR_SET_REF &&
-	    expr->mappings->set->datatype->type == TYPE_VERDICT)
+	    expr->mappings->set->data->dtype->type == TYPE_VERDICT)
 		type = "vmap";
 
 	return json_pack("{s:{s:o, s:o}}", type,
src/mnl.c +3 -3
index 9c1f5356c9b9..034f21709a19 100644
@@ -815,9 +815,9 @@ int mnl_nft_set_add(struct netlink_ctx *ctx, const struct cmd *cmd,
 			  div_round_up(set->key->len, BITS_PER_BYTE));
 	if (set_is_datamap(set->flags)) {
 		nftnl_set_set_u32(nls, NFTNL_SET_DATA_TYPE,
-				  dtype_map_to_kernel(set->datatype));
+				  dtype_map_to_kernel(set->data->dtype));
 		nftnl_set_set_u32(nls, NFTNL_SET_DATA_LEN,
-				  set->datalen / BITS_PER_BYTE);
+				  set->data->len / BITS_PER_BYTE);
 	}
 	if (set_is_objmap(set->flags))
 		nftnl_set_set_u32(nls, NFTNL_SET_OBJ_TYPE, set->objtype);
@@ -849,7 +849,7 @@ int mnl_nft_set_add(struct netlink_ctx *ctx, const struct cmd *cmd,
 
 	if (set_is_datamap(set->flags) &&
 	    !nftnl_udata_put_u32(udbuf, NFTNL_UDATA_SET_DATABYTEORDER,
-				 set->datatype->byteorder))
+				 set->data->byteorder))
 		memory_allocation_error();
 
 	if (set->automerge &&
src/monitor.c +1 -1
index 40c381149cda..d90bbfd4d505 100644
@@ -401,7 +401,7 @@ static int netlink_events_setelem_cb(const struct nlmsghdr *nlh, int type,
 	 */
 	dummyset = set_alloc(monh->loc);
 	dummyset->key = expr_clone(set->key);
-	dummyset->datatype = set->datatype;
+	dummyset->data = set->data;
 	dummyset->flags = set->flags;
 	dummyset->init = set_expr_alloc(monh->loc, set);
 
src/netlink.c +15 -17
index f8e1120447d9..773b98cec61a 100644
@@ -520,7 +520,7 @@ enum nft_data_types dtype_map_to_kernel(const struct datatype *dtype)
 	}
 }
 
-const struct datatype *dtype_map_from_kernel(enum nft_data_types type)
+static const struct datatype *dtype_map_from_kernel(enum nft_data_types type)
 {
 	switch (type) {
 	case NFT_DATA_VERDICT:
@@ -567,10 +567,10 @@ struct set *netlink_delinearize_set(struct netlink_ctx *ctx,
 				    const struct nftnl_set *nls)
 {
 	const struct nftnl_udata *ud[NFTNL_UDATA_SET_MAX + 1] = {};
-	uint32_t flags, key, data, data_len, objtype = 0;
 	enum byteorder keybyteorder = BYTEORDER_INVALID;
 	enum byteorder databyteorder = BYTEORDER_INVALID;
-	const struct datatype *keytype, *datatype;
+	const struct datatype *keytype, *datatype = NULL;
+	uint32_t flags, key, objtype = 0;
 	bool automerge = false;
 	const char *udata;
 	struct set *set;
@@ -604,6 +604,8 @@ struct set *netlink_delinearize_set(struct netlink_ctx *ctx,
 
 	flags = nftnl_set_get_u32(nls, NFTNL_SET_FLAGS);
 	if (set_is_datamap(flags)) {
+		uint32_t data;
+
 		data = nftnl_set_get_u32(nls, NFTNL_SET_DATA_TYPE);
 		datatype = dtype_map_from_kernel(data);
 		if (datatype == NULL) {
@@ -612,8 +614,7 @@ struct set *netlink_delinearize_set(struct netlink_ctx *ctx,
 					 data);
 			return NULL;
 		}
-	} else
-		datatype = NULL;
+	}
 
 	if (set_is_objmap(flags)) {
 		objtype = nftnl_set_get_u32(nls, NFTNL_SET_OBJ_TYPE);
@@ -636,16 +637,13 @@ struct set *netlink_delinearize_set(struct netlink_ctx *ctx,
 
 	set->objtype = objtype;
 
+	set->data = NULL;
 	if (datatype)
-		set->datatype = datatype_get(set_datatype_alloc(datatype,
-								databyteorder));
-	else
-		set->datatype = NULL;
-
-	if (nftnl_set_is_set(nls, NFTNL_SET_DATA_LEN)) {
-		data_len = nftnl_set_get_u32(nls, NFTNL_SET_DATA_LEN);
-		set->datalen = data_len * BITS_PER_BYTE;
-	}
+		set->data = constant_expr_alloc(&netlink_location,
+						set_datatype_alloc(datatype, databyteorder),
+						databyteorder,
+						nftnl_set_get_u32(nls, NFTNL_SET_DATA_LEN) * BITS_PER_BYTE,
+						NULL);
 
 	if (nftnl_set_is_set(nls, NFTNL_SET_TIMEOUT))
 		set->timeout = nftnl_set_get_u64(nls, NFTNL_SET_TIMEOUT);
@@ -830,10 +828,10 @@ int netlink_delinearize_setelem(struct nftnl_set_elem *nlse,
 			goto out;
 
 		data = netlink_alloc_data(&netlink_location, &nld,
-					  set->datatype->type == TYPE_VERDICT ?
+					  set->data->dtype->type == TYPE_VERDICT ?
 					  NFT_REG_VERDICT : NFT_REG_1);
-		datatype_set(data, set->datatype);
-		data->byteorder = set->datatype->byteorder;
+		datatype_set(data, set->data->dtype);
+		data->byteorder = set->data->byteorder;
 		if (data->byteorder == BYTEORDER_HOST_ENDIAN)
 			mpz_switch_byteorder(data->value, data->len / BITS_PER_BYTE);
 
src/parser_bison.y +1 -2
index bff5e274c787..c531ee1d1dd8 100644
@@ -1690,9 +1690,8 @@ map_block		:	/* empty */	{ $$ = $<set>-1; }
 						stmt_separator
 			{
 				$1->key = $3;
-				$1->datatype = $5->dtype;
+				$1->data = $5;
 
-				expr_free($5);
 				$1->flags |= NFT_SET_MAP;
 				$$ = $1;
 			}
src/parser_json.c +6 -2
index a969bd4c3676..fbb90da60a83 100644
@@ -2791,11 +2791,15 @@ static struct cmd *json_parse_cmd_add_set(struct json_ctx *ctx, json_t *root,
 	}
 
 	if (!json_unpack(root, "{s:s}", "map", &dtype_ext)) {
+		const struct datatype *dtype;
+
 		set->objtype = string_to_nft_object(dtype_ext);
 		if (set->objtype) {
 			set->flags |= NFT_SET_OBJECT;
-		} else if (datatype_lookup_byname(dtype_ext)) {
-			set->datatype = datatype_lookup_byname(dtype_ext);
+		} else if ((dtype = datatype_lookup_byname(dtype_ext))) {
+			set->data = constant_expr_alloc(&netlink_location,
+							dtype, dtype->byteorder,
+							dtype->size, NULL);
 			set->flags |= NFT_SET_MAP;
 		} else {
 			json_error(ctx, "Invalid map type '%s'.", dtype_ext);
src/rule.c +4 -4
index 5655e8c011e4..aee08ea12c8b 100644
@@ -330,8 +330,8 @@ struct set *set_clone(const struct set *set)
 	new_set->gc_int		= set->gc_int;
 	new_set->timeout	= set->timeout;
 	new_set->key		= expr_clone(set->key);
-	new_set->datatype	= datatype_get(set->datatype);
-	new_set->datalen	= set->datalen;
+	if (set->data)
+		new_set->data	= expr_clone(set->data);
 	new_set->objtype	= set->objtype;
 	new_set->policy		= set->policy;
 	new_set->automerge	= set->automerge;
@@ -354,7 +354,7 @@ void set_free(struct set *set)
 		expr_free(set->init);
 	handle_free(&set->handle);
 	expr_free(set->key);
-	set_datatype_destroy(set->datatype);
+	expr_free(set->data);
 	xfree(set);
 }
 
@@ -468,7 +468,7 @@ static void set_print_declaration(const struct set *set,
 	nft_print(octx, "%s%stype %s",
 		  opts->tab, opts->tab, set->key->dtype->name);
 	if (set_is_datamap(set->flags))
-		nft_print(octx, " : %s", set->datatype->name);
+		nft_print(octx, " : %s", set->data->dtype->name);
 	else if (set_is_objmap(set->flags))
 		nft_print(octx, " : %s", obj_type_name(set->objtype));
 
src/segtree.c +6 -2
index eff0653a8dfb..cfb892c3bb34 100644
@@ -79,8 +79,12 @@ static void seg_tree_init(struct seg_tree *tree, const struct set *set,
 	tree->root	= RB_ROOT;
 	tree->keytype	= set->key->dtype;
 	tree->keylen	= set->key->len;
-	tree->datatype	= set->datatype;
-	tree->datalen	= set->datalen;
+	tree->datatype	= NULL;
+	tree->datalen	= 0;
+	if (set->data) {
+		tree->datatype	= set->data->dtype;
+		tree->datalen	= set->data->len;
+	}
 	tree->byteorder	= first->byteorder;
 	tree->debug_mask = debug_mask;
 }
Signature · 1 line
2.21.0
author maintainer
#4 lore raw

PATCH 4/8 src: parser: add syntax to provide bitsize for non-spcific types

In reply to Florian Westphal

This allows creation of sets with string and integer types by providing the datatype width using a comma, e.g.

"type string, 64" or "integer, 32".

This is mainly intended as a fallback for the upcoming "typeof" keyword -- if we can't make sense of the kernel provided type (or its missing entirely), we can then fallback to this format.

---
 src/parser_bison.y | 18 ++++++++++++++++++
 src/rule.c         | 24 +++++++++++++++++++-----
 2 files changed, 37 insertions(+), 5 deletions(-)
src/parser_bison.y +18 -0
index c531ee1d1dd8..ee169fbac194 100644
@@ -1826,6 +1826,24 @@ data_type_atom_expr	:	type_identifier
 							 dtype->size, NULL);
 				xfree($1);
 			}
+			|	type_identifier	COMMA	NUM
+			{
+				const struct datatype *dtype = datatype_lookup_byname($1);
+				if (dtype == NULL) {
+					erec_queue(error(&@1, "unknown datatype %s", $1),
+						   state->msgs);
+					YYERROR;
+				}
+
+				if (dtype->size) {
+					erec_queue(error(&@1, "Datatype %s has a fixed type", $1),
+						   state->msgs);
+					YYERROR;
+				}
+				$$ = constant_expr_alloc(&@1, dtype, dtype->byteorder,
+							 $3, NULL);
+				xfree($1);
+			}
 			;
 
 data_type_expr		:	data_type_atom_expr
src/rule.c +19 -5
index aee08ea12c8b..59369c9082a3 100644
@@ -436,6 +436,16 @@ const char *set_policy2str(uint32_t policy)
 	}
 }
 
+static void set_print_key(const struct expr *expr, struct output_ctx *octx)
+{
+	const struct datatype *dtype = expr->dtype;
+
+	if (dtype->size)
+		nft_print(octx, " %s", dtype->name);
+	else
+		nft_print(octx, " %s,%d", dtype->name, expr->len);
+}
+
 static void set_print_declaration(const struct set *set,
 				  struct print_fmt_options *opts,
 				  struct output_ctx *octx)
@@ -465,12 +475,16 @@ static void set_print_declaration(const struct set *set,
 	if (nft_output_handle(octx))
 		nft_print(octx, " # handle %" PRIu64, set->handle.handle.id);
 	nft_print(octx, "%s", opts->nl);
-	nft_print(octx, "%s%stype %s",
-		  opts->tab, opts->tab, set->key->dtype->name);
-	if (set_is_datamap(set->flags))
-		nft_print(octx, " : %s", set->data->dtype->name);
-	else if (set_is_objmap(set->flags))
+	nft_print(octx, "%s%stype ",
+		  opts->tab, opts->tab);
+	set_print_key(set->key, octx);
+
+	if (set_is_datamap(set->flags)) {
+		nft_print(octx, " : ");
+		set_print_key(set->data, octx);
+	} else if (set_is_objmap(set->flags)) {
 		nft_print(octx, " : %s", obj_type_name(set->objtype));
+	}
 
 	nft_print(octx, "%s", opts->stmt_separator);
 
Signature · 1 line
2.21.0
author maintainer
#5 lore raw

PATCH 5/8 src: add "typeof" keyword

In reply to Florian Westphal

This allows users to specify named sets by using the expression directly, rather than having to lookup the data type to use, or the needed size via 'nft describe".

Example:

table filter {
    set allowed_dports {
        type typeof(tcp dport);
    }
    map nametomark {
        type typeof(osf name) : typeof(meta mark);
    }
    map port2helper {
        type ipv4_addr . inet_service : typeof(ct helper);
    }
}

Currently, listing such a table will lose the typeof() expression:

nft will print the datatype instead, just as if "type inet_service" would have been used.

For types with non-fixed widths, the new "type, width" format added in previous patch is used.

---
 src/parser_bison.y | 5 +++++
 src/scanner.l      | 1 +
 2 files changed, 6 insertions(+)
src/parser_bison.y +5 -0
index ee169fbac194..876050ba6863 100644
@@ -192,6 +192,7 @@ int nft_lex(void *, void *, void *);
 %token DEFINE			"define"
 %token REDEFINE			"redefine"
 %token UNDEFINE			"undefine"
+%token TYPEOF			"typeof"
 
 %token FIB			"fib"
 
@@ -1844,6 +1845,10 @@ data_type_atom_expr	:	type_identifier
 							 $3, NULL);
 				xfree($1);
 			}
+			|	TYPEOF	'('	primary_expr	')'
+			{
+				$$ = $3;
+			}
 			;
 
 data_type_expr		:	data_type_atom_expr
src/scanner.l +1 -0
index c1adcbddbd73..cd563aa0ca1f 100644
@@ -243,6 +243,7 @@ addrstring	({macaddr}|{ip4addr}|{ip6addr})
 "define"		{ return DEFINE; }
 "redefine"		{ return REDEFINE; }
 "undefine"		{ return UNDEFINE; }
+"typeof"		{ return TYPEOF; }
 
 "describe"		{ return DESCRIBE; }
 
Signature · 1 line
2.21.0
author maintainer
#6 lore raw

PATCH 6/8 src: add "typeof" print support

In reply to Florian Westphal

Use the 'udata' facility to store the original string in the kernel, then use the existing 'describe' facility to resolve it back to a name.

This isn't strictly required, in case resolution fails, the udata is inconsistent (e.g., because set specifies a size that is not identical to the expressions size that we get when decoding the expression), if the udata is missing, etc. then the "type, width" format is used.

This allows us to list and restore such sets even if we can't reconstruct the original expression.

---
 src/libnftables.c |   2 +-
 src/mnl.c         |  33 ++++++++++
 src/netlink.c     | 156 ++++++++++++++++++++++++++++++++++++++++++----
 src/rule.c        |  13 ++--
 4 files changed, 186 insertions(+), 18 deletions(-)
src/libnftables.c +1 -1
index a1e2fd662a7a..e9ebc3f389fc 100644
@@ -424,7 +424,7 @@ static int nft_evaluate(struct nft_ctx *nft, struct list_head *msgs,
 			.msgs	= msgs,
 		};
 		if (cmd_evaluate(&ectx, cmd) < 0 &&
-		    ++nft->state->nerrs == nft->parser_max_errors)
+		    ++nft->state->nerrs >= nft->parser_max_errors)
 			return -1;
 	}
 
src/mnl.c +33 -0
index 034f21709a19..5e5f3628cfe1 100644
@@ -787,6 +787,35 @@ err:
 	return NULL;
 }
 
+static void set_key_expression(struct netlink_ctx *ctx,
+				struct expr *expr, uint32_t set_flags,
+				struct nftnl_udata_buf *udbuf,
+				unsigned int type)
+{
+	struct output_ctx octx = {};
+	char buf[64];
+
+	if (expr->flags & EXPR_F_CONSTANT ||
+	    set_flags & NFT_SET_ANONYMOUS)
+		return;
+
+	buf[0] = 0;
+	/* Set definition uses typeof to define datatype. */
+	octx.output_fp = fmemopen(buf, sizeof(buf), "w");
+	if (octx.output_fp) {
+		char *end;
+
+		expr_print(expr, &octx);
+		fclose(octx.output_fp);
+		end = strchr(buf, '&');
+		if (end)
+			* end = 0;
+
+		if (!nftnl_udata_put(udbuf, type, strlen(buf) + 1, buf))
+			memory_allocation_error();
+	}
+}
+
 /*
  * Set
  */
@@ -857,6 +886,10 @@ int mnl_nft_set_add(struct netlink_ctx *ctx, const struct cmd *cmd,
 				 set->automerge))
 		memory_allocation_error();
 
+	set_key_expression(ctx, set->key, set->flags, udbuf, NFTNL_UDATA_SET_TYPEOF_KEY);
+	if (set->data)
+		set_key_expression(ctx, set->data, set->flags, udbuf, NFTNL_UDATA_SET_TYPEOF_DATA);
+
 	nftnl_set_set_data(nls, NFTNL_SET_USERDATA, nftnl_udata_buf_data(udbuf),
 			   nftnl_udata_buf_len(udbuf));
 	nftnl_udata_buf_free(udbuf);
src/netlink.c +143 -13

214 lines not on this page · raw message

src/rule.c +9 -4
index 59369c9082a3..db9401c6a035 100644
@@ -440,10 +440,15 @@ static void set_print_key(const struct expr *expr, struct output_ctx *octx)
 {
 	const struct datatype *dtype = expr->dtype;
 
-	if (dtype->size)
-		nft_print(octx, " %s", dtype->name);
-	else
-		nft_print(octx, " %s,%d", dtype->name, expr->len);
+	if (dtype->size || dtype->type == TYPE_VERDICT)
+		nft_print(octx, "%s", dtype->name);
+	else if (expr->flags & EXPR_F_CONSTANT)
+		nft_print(octx, "%s,%d", dtype->name, expr->len);
+	else {
+		nft_print(octx, "typeof(");
+		expr_print(expr, octx);
+		nft_print(octx, ")");
+	}
 }
 
 static void set_print_declaration(const struct set *set,
Signature · 1 line
2.21.0
author maintainer
#7 lore raw

PATCH 7/8 src: netlink: remove assertion

In reply to Florian Westphal

This assert can trigger as follows:

set s {
	type integer,8
	elemets = { 1 }
};

vlan id @s accept

reason is that 'vlan id' will store a 16 bit value into the dreg, so set should use 'integer,16'.

The kernel won't detect this, as the lookup expression will only verify that it can load one byte from the given register.

This removes the assertion, in case we hit this condition we can just return without doing any further actions.

---
 src/netlink_delinearize.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
src/netlink_delinearize.c +13 -2
index fc2574b1dea9..11c2e2d87c54 100644
@@ -1796,9 +1796,20 @@ static void binop_adjust_one(const struct expr *binop, struct expr *value,
 {
 	struct expr *left = binop->left;
 
-	assert(value->len >= binop->right->len);
-
 	mpz_rshift_ui(value->value, shift);
+
+	/* This will happen when a set has a key that is
+	 * smaller than the amount of bytes loaded by the
+	 * payload/exthdr expression.
+	 *
+	 * This can't happen with normal nft frontend,
+	 * but it can happen with custom clients or with
+	 * nft sets defined via 'type integer,8' and then
+	 * asking "vlan id @myset".
+	 */
+	if (value->len < binop->right->len)
+		return;
+
 	switch (left->etype) {
 	case EXPR_PAYLOAD:
 	case EXPR_EXTHDR:
Signature · 1 line
2.21.0
author maintainer
#8 lore raw

PATCH 8/8 tests: add typeof test cases

In reply to Florian Westphal
---
 .../testcases/maps/dumps/typeof_maps_0.nft    | 16 ++++++++
 tests/shell/testcases/maps/typeof_maps_0      | 26 ++++++++++++
 .../testcases/sets/dumps/typeof_sets_0.nft    | 31 ++++++++++++++
 tests/shell/testcases/sets/typeof_sets_0      | 40 +++++++++++++++++++
 4 files changed, 113 insertions(+)
 create mode 100644 tests/shell/testcases/maps/dumps/typeof_maps_0.nft
 create mode 100755 tests/shell/testcases/maps/typeof_maps_0
 create mode 100644 tests/shell/testcases/sets/dumps/typeof_sets_0.nft
 create mode 100755 tests/shell/testcases/sets/typeof_sets_0
tests/shell/testcases/maps/dumps/typeof_maps_0.nft new file +16 -0
new file mode 100644
index 000000000000..d691f406047f
@@ -0,0 +1,16 @@
+table inet t {
+	map m1 {
+		type typeof(osf name) : mark
+		elements = { "Linux" : 0x00000001 }
+	}
+
+	map m2 {
+		type string,128 : mark
+		elements = { "Linux" : 0x00000001 }
+	}
+
+	chain c {
+		ct mark set osf name map @m1
+		ct mark set osf name map @m2
+	}
+}
tests/shell/testcases/maps/typeof_maps_0 new file +26 -0
new file mode 100755
index 000000000000..59112c11e224
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+# support for strings/typeof in named maps.
+# m1 and m2 are identical, they just use different
+# ways for declaration.
+
+EXPECTED="table inet t {
+	map m1 {
+		type typeof(osf name) : typeof(ct mark)
+		elements = { \"Linux\" : 0x1 }
+	}
+
+	map m2 {
+		type string, 128 : mark
+		elements = { \"Linux\" : 0x1 }
+	}
+
+	chain c {
+		ct mark set osf name map @m1
+		ct mark set osf name map @m2
+	}
+}"
+
+set -e
+$NFT -f - <<< $EXPECTED
+
tests/shell/testcases/sets/dumps/typeof_sets_0.nft new file +31 -0
new file mode 100644
index 000000000000..7f588fc89ab7
@@ -0,0 +1,31 @@
+table inet t {
+	set s1 {
+		type typeof(osf name)
+		elements = { "Linux" }
+	}
+
+	set s2 {
+		type string,128
+		elements = { "Linux" }
+	}
+
+	set s3 {
+		type typeof(vlan id)
+		elements = { 2, 3, 103 }
+	}
+
+	set s4 {
+		type integer,16
+		elements = { 2, 3, 103, 2003 }
+	}
+
+	chain c1 {
+		osf name @s1 accept
+		osf name @s2 accept
+	}
+
+	chain c2 {
+		vlan id @s3 accept
+		vlan id @s4 accept
+	}
+}
tests/shell/testcases/sets/typeof_sets_0 new file +40 -0
new file mode 100755
index 000000000000..6ffa10107727
@@ -0,0 +1,40 @@
+#!/bin/bash
+
+# support for strings/typeof in named sets.
+# s1 and s2 are identical, they just use different
+# ways for declaration.
+
+EXPECTED="table inet t {
+	set s1 {
+		type typeof(osf name)
+		elements = { \"Linux\" }
+	}
+
+	set s2 {
+		type string, 128
+		elements = { \"Linux\" }
+	}
+
+	set s3 {
+		type typeof(vlan id)
+		elements = { 2, 3, 103 }
+	}
+
+	set s4 {
+		type integer,16
+		elements = { 2, 3, 103, 2003 }
+	}
+	chain c1 {
+		osf name @s1 accept
+		osf name @s2 accept
+	}
+
+	chain c2 {
+		ether type vlan vlan id @s3 accept
+		ether type vlan vlan id @s4 accept
+	}
+}"
+
+set -e
+$NFT -f - <<< $EXPECTED
+
Signature · 1 line
2.21.0
#9 lore raw
In reply to Florian Westphal

Hi Florian,

Quoted 4 lines · Florian Westphal

This patch series adds the typeof keyword.

The only dependency is a small change to libnftnl to add two new UDATA_SET_TYPEOF enum values.

Thanks for working on this.

Quoted 9 lines

named set can be configured as follows:

set os {
type typeof(osf name)
elements = { "Linux", "Windows" }
}

or
nft add set ip filter allowed "{ type typeof(ip daddr) . typeof(tcp dport); }"

I know I sent a RFC using typeof(), I wonder if you could just use the selector instead, it's a bit of a lot of type typeof() . typeof() probably.

So this is left as this:

type osf name

in concatenations, like this:

nft add set ip filter allowed "{ type ip daddr . tcp dport; }"

Probably I would ask my sysadmin friends what they think. I spent too much time on coding, so all these typeof() look natural to me, but it might be a bit too much syntactic sugar for someone that is more in network operations, not sure.

P.S: patch 1/8 and 2/8 are related to this patchset? After quick glance, not obvious to me or if they are again related to multiple nft_ctx_new() calls.

#10 lore raw

On Sat, Aug 17, 2019 at 12:23:51PM +0200, Pablo Neira Ayuso wrote:
[...]

Quoted 1 line · Florian Westphal

[..]

Quoted 3 lines

P.S: patch 1/8 and 2/8 are related to this patchset? After quick glance, not obvious to me or if they are again related to multiple nft_ctx_new() calls.

I found it, it is in patch 6/8, it making a call to set_make_key to get the datatype to translate the selector to datatype using the parser API.

author maintainer
#11 lore raw
Quoted 11 lines · Pablo Neira Ayuso

On Sat, Aug 17, 2019 at 12:23:51PM +0200, Pablo Neira Ayuso wrote:
[...]

Quoted 1 line · Florian Westphal

[..]

Quoted 3 lines

P.S: patch 1/8 and 2/8 are related to this patchset? After quick glance, not obvious to me or if they are again related to multiple nft_ctx_new() calls.

I found it, it is in patch 6/8, it making a call to set_make_key to get the datatype to translate the selector to datatype using the parser API.

Yes, I could push patch #1 independently though.

author maintainer
#12 lore raw
In reply to Pablo Neira Ayuso
Quoted 6 lines · Pablo Neira Ayuso

I know I sent a RFC using typeof(), I wonder if you could just use the selector instead, it's a bit of a lot of type typeof() . typeof() probably.

So this is left as this:

Quoted 7 lines

type osf name

in concatenations, like this:

nft add set ip filter allowed "{ type ip daddr . tcp dport; }"

Probably I would ask my sysadmin friends what they think.

Yes, please do, it would be good to get a non-developer perspective.

I'm very used to things like sizeof(), so typeof() felt natural to me.

Might be very un-intuitive for non-developers though, so it would be good to get outside perspective.

Thanks!

#13 lore raw
Quoted 13 lines · Florian Westphal
Quoted 12 lines · Pablo Neira Ayuso

I know I sent a RFC using typeof(), I wonder if you could just use the selector instead, it's a bit of a lot of type typeof() . typeof() probably.

So this is left as this:

type osf name

in concatenations, like this:

nft add set ip filter allowed "{ type ip daddr . tcp dport; }"

Quoted 9 lines
Quoted 1 line

Probably I would ask my sysadmin friends what they think.

Yes, please do, it would be good to get a non-developer perspective.

I'm very used to things like sizeof(), so typeof() felt natural to me.

Might be very un-intuitive for non-developers though, so it would be good to get outside perspective.

From my point of view, this is a rather advanced operation. As long as it is properly documented, I don't see any problem with typeof().

Also, just typeof would work of course. Up to you.

Thanks for working on this!

#14 lore raw

Hi Florian,

Quoted 20 lines · Arturo Borrero Gonzalez
Quoted 19 lines · Florian Westphal
Quoted 6 lines · Pablo Neira Ayuso

I know I sent a RFC using typeof(), I wonder if you could just use the selector instead, it's a bit of a lot of type typeof() . typeof() probably.

So this is left as this:

Quoted 7 lines

type osf name

in concatenations, like this:

nft add set ip filter allowed "{ type ip daddr . tcp dport; }"

Probably I would ask my sysadmin friends what they think.

Yes, please do, it would be good to get a non-developer perspective.

I'm very used to things like sizeof(), so typeof() felt natural to me.

Quoted 7 lines
Quoted 2 lines

Might be very un-intuitive for non-developers though, so it would be good to get outside perspective.

From my point of view, this is a rather advanced operation. As long as it is properly documented, I don't see any problem with typeof().

Also, just typeof would work of course. Up to you.

My suggestion is:

typeof ip saddr . tcp dport . meta mark

but, not to allow the use of the existing datatypes with typeof.

So either the user picks:

type ipv4_addr . inet_service . mark

or this:

typeof ip saddr . tcp dport . meta mark

Then, for the notation:

integer,bits

I would suggest this one can only be used from 'type'.

Goal is basically:

* Avoiding repetitive typeof() that looks a bit like C programming.
* Mixing datatype with inferred datatypes looks a bit sloppy to me from syntax consistency point of view.

This approach is flexible enough to cover all use cases this patchset is dealing with I think.

If you like this approach, I would suggest you just update the patchset to support this and then push this out, I'll catch up later on to revisit this before the next release, in case I can propose you some refinement for your review.

Let me know, thanks.