Skip to contentSkip to conversation

add description infrastructure

PATCH In review Pablo Neira Ayuso 2 participants 11 messages Jan 20, 2022 – Apr 6, 2022 netfilter

lore
Share & tools
lore

Pablo Neira Ayuso

author maintainer
#0 lore raw

cover add description infrastructure

Hi Phil,

This is my proposal to address the snprintf data printing depending on the arch. The idea is to add description objects that can be used to build the userdata area as well as to parse the userdata to create the description object.

This is revisiting 6e48df5329ea ("src: add "typeof" build/parse/print support") in nftables which adds build and parse userdata callbacks to expression in libnftables. My proposal is to move this to libnftnl.

This allows to consolidate codebase to address two different usecases:
- you can pass the description object to the snprintf function.
- it provides helpers to build and to parse the userdata area. The userdata TLV attributes do not need to be exposed through libnftnl/udata.h anymore, instead users can just rely on these helper functions.

The userdata area has been extended with new attributes, but it is still incomplete since it does not allow to represent a concatenation.

Note that this will also allow us to deprecate NFTA_SET_DATA_TYPE at some point (this netlink attribute represents a concatenation using 6 bits of the 32-bit integer, hence limiting concatenations to 5 components at this stage).

I'm afraid we'll also have to keep the existing userdata attributes added by 6e48df5329ea in nftables for a little while (at least the parser functions), so nftables does not break on updates since my userdata TLV coming in this patchset for libnftnl is different than the one available at 6e48df5329ea.

Please also note that nftables needs to be updated to use this infrastructure.

My proposal follows a longer route but it will allow to addressing a number of existing shortcomings in the set infrastrcture.

This is compile-tested only.

Pablo Neira Ayuso (3):
desc: add expression description
desc: add datatype description
desc: add set description

 include/Makefile.am          |   1 +
 include/desc.h               |  50 +++
 include/expr_ops.h           |  11 +
 include/internal.h           |   1 +
 include/libnftnl/Makefile.am |   1 +
 include/libnftnl/desc.h      | 107 +++++++
 include/libnftnl/udata.h     |  18 +-
 src/Makefile.am              |   1 +
 src/desc.c                   | 598 +++++++++++++++++++++++++++++++++++
 src/expr/payload.c           |  81 +++++
 src/expr_ops.c               |  13 +
 11 files changed, 875 insertions(+), 7 deletions(-)
 create mode 100644 include/desc.h
 create mode 100644 include/libnftnl/desc.h
 create mode 100644 src/desc.c
Signature · 1 line
2.30.2
author maintainer
#1 lore raw

PATCH 1/3 desc: add expression description

Add a new object to describe an expression. This allows to describe the set key when typeof is used to define the set.

---
 include/Makefile.am          |   1 +
 include/desc.h               |  19 +++++
 include/expr_ops.h           |  11 +++
 include/internal.h           |   1 +
 include/libnftnl/Makefile.am |   1 +
 include/libnftnl/desc.h      |  57 ++++++++++++++
 src/Makefile.am              |   1 +
 src/desc.c                   | 142 +++++++++++++++++++++++++++++++++++
 src/expr/payload.c           |  81 ++++++++++++++++++++
 src/expr_ops.c               |  13 ++++
 10 files changed, 327 insertions(+)
 create mode 100644 include/desc.h
 create mode 100644 include/libnftnl/desc.h
 create mode 100644 src/desc.c
include/Makefile.am +1 -0
index 738f80708ca0..83a0e5209ed7 100644
@@ -3,6 +3,7 @@ SUBDIRS = libnftnl linux
 noinst_HEADERS = internal.h	\
 		 linux_list.h	\
 		 data_reg.h	\
+		 desc.h		\
 		 expr_ops.h	\
 		 obj.h		\
 		 linux_list.h	\
include/desc.h new file +19 -0
new file mode 100644
index 000000000000..d78af3528118
@@ -0,0 +1,19 @@
+#ifndef _LIBNFTNL_DESC_INTERNAL_H_
+#define _LIBNFTNL_DESC_INTERNAL_H_
+
+struct nftnl_expr_desc {
+	uint32_t	etype;
+	uint32_t	byteorder;
+	uint32_t	len;
+
+	struct expr_ops *ops;
+
+	union {
+		struct {
+			uint32_t desc_id;
+			uint32_t type;
+		} payload;
+	};
+};
+
+#endif
include/expr_ops.h +11 -0
index 7a6aa23f9bd1..77e010bf2a7d 100644
@@ -2,14 +2,19 @@
 #define _EXPR_OPS_H_
 
 #include <stdint.h>
+#include <libnftnl/desc.h>
 #include "internal.h"
 
 struct nlattr;
 struct nlmsghdr;
 struct nftnl_expr;
+struct nftnl_expr_desc;
+struct nftnl_udata;
+struct nftnl_udata_buf;
 
 struct expr_ops {
 	const char *name;
+	enum nftnl_expr_desc_type type;
 	uint32_t alloc_len;
 	int	max_attr;
 	void	(*init)(const struct nftnl_expr *e);
@@ -19,9 +24,15 @@ struct expr_ops {
 	int 	(*parse)(struct nftnl_expr *e, struct nlattr *attr);
 	void	(*build)(struct nlmsghdr *nlh, const struct nftnl_expr *e);
 	int	(*snprintf)(char *buf, size_t len, uint32_t flags, const struct nftnl_expr *e);
+	struct {
+		int	(*set)(struct nftnl_expr_desc *uexpr, uint8_t type, const void *data, uint32_t data_len);
+		int	(*build)(struct nftnl_udata_buf *ud, const struct nftnl_expr_desc *uexpr);
+		int	(*parse)(const struct nftnl_udata *attr, struct nftnl_expr_desc *uexpr);
+	} desc;
 };
 
 struct expr_ops *nftnl_expr_ops_lookup(const char *name);
+struct expr_ops *nftnl_expr_ops_lookup_by_type(uint32_t type);
 
 #define nftnl_expr_data(ops) (void *)ops->data
 
include/internal.h +1 -0
index 1f96731589c0..7b058aa2ced8 100644
@@ -12,5 +12,6 @@
 #include "expr.h"
 #include "expr_ops.h"
 #include "rule.h"
+#include "desc.h"
 
 #endif /* _LIBNFTNL_INTERNAL_H_ */
include/libnftnl/Makefile.am +1 -0
index d846a574f438..3d072ed8642b 100644
@@ -1,4 +1,5 @@
 pkginclude_HEADERS = batch.h		\
+		     desc.h		\
 		     table.h		\
 		     trace.h		\
 		     chain.h		\
include/libnftnl/desc.h new file +57 -0

57 lines not on this page · raw message

src/Makefile.am +1 -0
index c3b0ab974bd2..06e1567ffd13 100644
@@ -7,6 +7,7 @@ libnftnl_la_LDFLAGS = -Wl,--version-script=$(srcdir)/libnftnl.map	\
 
 libnftnl_la_SOURCES = utils.c		\
 		      batch.c		\
+		      desc.c		\
 		      flowtable.c	\
 		      common.c		\
 		      gen.c		\
src/desc.c new file +142 -0

142 lines not on this page · raw message

src/expr/payload.c +81 -0

99 lines not on this page · raw message

src/expr_ops.c +13 -0
index 7248e4f98b0a..383414a42619 100644
@@ -102,3 +102,16 @@ struct expr_ops *nftnl_expr_ops_lookup(const char *name)
 	}
 	return NULL;
 }
+
+struct expr_ops *nftnl_expr_ops_lookup_by_type(uint32_t type)
+{
+	int i = 0;
+
+	while (expr_ops[i] != NULL) {
+		if (expr_ops[i]->type == type)
+			return expr_ops[i];
+
+		i++;
+	}
+	return NULL;
+}
Signature · 1 line
2.30.2
author maintainer
#2 lore raw

PATCH 2/3 desc: add datatype description

In reply to Pablo Neira Ayuso

Add a new object to describe a datatype. This allows to describe the set key when type is used to define the set.

---
 include/desc.h          |  6 +++
 include/libnftnl/desc.h | 19 ++++++++
 src/desc.c              | 96 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 121 insertions(+)
include/desc.h +6 -0
index d78af3528118..2f61a1e5963e 100644
@@ -16,4 +16,10 @@ struct nftnl_expr_desc {
 	};
 };
 
+struct nftnl_dtype_desc {
+	uint32_t	dtype;
+	uint32_t	byteorder;
+	uint32_t	len;
+};
+
 #endif
include/libnftnl/desc.h +19 -0
index 1202eb9ddf79..cb1cac91f934 100644
@@ -50,6 +50,25 @@ int nftnl_expr_desc_build(struct nftnl_udata_buf *udbuf,
 int nftnl_expr_desc_parse(const struct nftnl_udata *attr,
 			  struct nftnl_expr_desc *dexpr);
 
+enum nftnl_dtype_desc_types {
+	NFTNL_DESC_DTYPE_TYPE,
+	NFTNL_DESC_DTYPE_BYTEORDER,
+	NFTNL_DESC_DTYPE_LEN,
+	__NFTNL_DESC_DTYPE_MAX
+};
+#define NFTNL_DESC_DTYPE_MAX (__NFTNL_DESC_DTYPE_MAX - 1)
+
+struct nftnl_dtype_desc;
+
+struct nftnl_dtype_desc *nftnl_dtype_desc_alloc(void);
+void nftnl_dtype_desc_free(struct nftnl_dtype_desc *dtype);
+int nftnl_dtype_desc_set(struct nftnl_dtype_desc *dtype, uint8_t type,
+			 const void *data, uint32_t data_len);
+int nftnl_dtype_desc_build(struct nftnl_udata_buf *udbuf,
+			   const struct nftnl_dtype_desc *dtype);
+int nftnl_dtype_desc_parse(const struct nftnl_udata *attr,
+			    struct nftnl_dtype_desc *dtype);
+
 #ifdef __cplusplus
 } /* extern "C" */
 #endif
src/desc.c +96 -0

99 lines not on this page · raw message

Signature · 1 line
2.30.2
author maintainer
#3 lore raw

PATCH 3/3 desc: add set description

In reply to Pablo Neira Ayuso

Add a new object to describe a set. Add helpers to build and to parse the userdata set description.

---
 include/desc.h           |  25 +++
 include/libnftnl/desc.h  |  31 ++++
 include/libnftnl/udata.h |  18 +-
 src/desc.c               | 360 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 427 insertions(+), 7 deletions(-)
include/desc.h +25 -0
index 2f61a1e5963e..135f1a7b6317 100644
@@ -1,6 +1,8 @@
 #ifndef _LIBNFTNL_DESC_INTERNAL_H_
 #define _LIBNFTNL_DESC_INTERNAL_H_
 
+#include <libnftnl/udata.h>
+
 struct nftnl_expr_desc {
 	uint32_t	etype;
 	uint32_t	byteorder;
@@ -22,4 +24,27 @@ struct nftnl_dtype_desc {
 	uint32_t	len;
 };
 
+#define NFTNL_SET_DESC_MAX        6
+
+struct nftnl_concat_desc {
+	union {
+		struct {
+			const struct nftnl_expr_desc	*expr[NFTNL_SET_DESC_MAX];
+			uint32_t			num_typeof;
+		};
+		struct {
+			const struct nftnl_dtype_desc	*dtype[NFTNL_SET_DESC_MAX];
+			uint32_t			num_type;
+		};
+	};
+};
+
+struct nftnl_set_desc {
+	enum nftnl_set_desc_type	type;
+	uint32_t			flags;
+	struct nftnl_concat_desc	key;
+	struct nftnl_concat_desc	data;
+	char				comment[NFTNL_UDATA_COMMENT_MAXLEN];
+};
+
 #endif
include/libnftnl/desc.h +31 -0
index cb1cac91f934..aa14145d4b41 100644
@@ -69,6 +69,37 @@ int nftnl_dtype_desc_build(struct nftnl_udata_buf *udbuf,
 int nftnl_dtype_desc_parse(const struct nftnl_udata *attr,
 			    struct nftnl_dtype_desc *dtype);
 
+enum nftnl_set_desc_type {
+	NFTNL_DESC_SET_UNSPEC	= 0,
+	NFTNL_DESC_SET_DATATYPE,
+	NFTNL_DESC_SET_TYPEOF,
+};
+
+enum nftnl_set_desc_types {
+	NFTNL_DESC_SET_TYPE,
+	NFTNL_DESC_SET_KEY,
+	NFTNL_DESC_SET_DATA,
+	NFTNL_DESC_SET_FLAGS,
+	NFTNL_DESC_SET_COMMENT,
+	__NFTNL_DESC_SET_MAX
+};
+#define NFTNL_DESC_SET_MAX (__NFTNL_DESC_SET_MAX - 1)
+
+struct nftnl_set_desc;
+
+struct nftnl_set_desc *nftnl_set_desc_alloc(void);
+void nftnl_set_desc_free(struct nftnl_set_desc *dset);
+int nftnl_set_desc_set(struct nftnl_set_desc *dset, enum nftnl_set_desc_types type,
+		       const void *data, uint32_t data_len);
+int nftnl_set_desc_add_expr(struct nftnl_set_desc *su, enum nftnl_set_desc_types type,
+			    const struct nftnl_expr_desc *dexpr);
+int nftnl_set_desc_add_datatype(struct nftnl_set_desc *dset, enum nftnl_set_desc_types type,
+				const struct nftnl_dtype_desc *dtype);
+int nftnl_set_desc_build_udata(struct nftnl_udata_buf *udbuf,
+			       const struct nftnl_set_desc *dset);
+int nftnl_set_desc_parse_udata(const char *udata, uint32_t len,
+			       struct nftnl_set_desc *dset);
+
 #ifdef __cplusplus
 } /* extern "C" */
 #endif
include/libnftnl/udata.h +11 -7
index dbf3a60ff06e..3349ae7bf7b1 100644
@@ -37,14 +37,18 @@ enum nftnl_udata_obj_types {
 #define NFTNL_UDATA_COMMENT_MAXLEN	128
 
 enum nftnl_udata_set_types {
-	NFTNL_UDATA_SET_KEYBYTEORDER,
-	NFTNL_UDATA_SET_DATABYTEORDER,
-	NFTNL_UDATA_SET_MERGE_ELEMENTS,
-	NFTNL_UDATA_SET_KEY_TYPEOF,
-	NFTNL_UDATA_SET_DATA_TYPEOF,
-	NFTNL_UDATA_SET_EXPR,
-	NFTNL_UDATA_SET_DATA_INTERVAL,
+	NFTNL_UDATA_SET_KEYBYTEORDER,	/* not used in newer versions */
+	NFTNL_UDATA_SET_DATABYTEORDER,	/* not used in newer versions */
+	NFTNL_UDATA_SET_MERGE_ELEMENTS,	/* not used in newer versions */
+	NFTNL_UDATA_SET_KEY_TYPEOF,	/* not used in newer versions */
+	NFTNL_UDATA_SET_DATA_TYPEOF,	/* not used in newer versions */
+	NFTNL_UDATA_SET_EXPR,		/* not used in newer versions */
+	NFTNL_UDATA_SET_DATA_INTERVAL,	/* not used in newer versions */
 	NFTNL_UDATA_SET_COMMENT,
+	NFTNL_UDATA_SET_TYPE,
+	NFTNL_UDATA_SET_KEY,
+	NFTNL_UDATA_SET_DATA,
+	NFTNL_UDATA_SET_FLAGS,
 	__NFTNL_UDATA_SET_MAX
 };
 #define NFTNL_UDATA_SET_MAX (__NFTNL_UDATA_SET_MAX - 1)
src/desc.c +360 -0

363 lines not on this page · raw message

Signature · 1 line
2.30.2
reviewer
#4 lore raw

Re: [PATCH libnftnl 3/3] desc: add set description

Hi,

On Thu, Jan 20, 2022 at 01:04:02AM +0100, Pablo Neira Ayuso wrote:
[...]

Quoted 4 lines
src/desc.c +0 -0
index f73e74c2c7d3..c8b3195db850 100644

[...]

Quoted 85 lines
src/desc.c +85 -0
+static int nftnl_set_desc_build_dtype(struct nftnl_udata_buf *udbuf,
+				      const struct nftnl_set_desc *dset)
+{
+	struct nftnl_udata *nest;
+	int i, err;
+
+	switch (dset->type) {
+	case NFTNL_DESC_SET_TYPEOF:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_KEY);
+		for (i = 0; i < dset->key.num_type; i++) {
+			err = __nftnl_udata_set_dtype_build(udbuf, dset->key.dtype[i], i);
+			if (err < 0)
+				return err;
+		}
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_DATATYPE:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_DATA);
+		for (i = 0; i < dset->data.num_type; i++) {
+			err = __nftnl_udata_set_dtype_build(udbuf, dset->data.dtype[i], i);
+			if (err < 0)
+				return err;
+		}
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_UNSPEC:
+		return -1;
+	}
+
+	return 0;
+}
+
+static int __nftnl_set_desc_build_typeof(struct nftnl_udata_buf *udbuf,
+					 const struct nftnl_expr_desc *dexpr,
+					 uint8_t attr_type)
+{
+	struct nftnl_udata *nest;
+	int err;
+
+	nest = nftnl_udata_nest_start(udbuf, attr_type);
+	err = nftnl_expr_desc_build(udbuf, dexpr);
+	nftnl_udata_nest_end(udbuf, nest);
+
+	return err;
+}
+
+static int nftnl_set_desc_build_typeof(struct nftnl_udata_buf *udbuf,
+				       const struct nftnl_set_desc *dset)
+{
+	struct nftnl_udata *nest;
+	int i;
+
+	switch (dset->type) {
+	case NFTNL_DESC_SET_TYPEOF:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_KEY_TYPEOF);
+		for (i = 0; i < dset->key.num_typeof; i++)
+			__nftnl_set_desc_build_typeof(udbuf, dset->key.expr[i], i);
+
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_DATATYPE:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_DATA_TYPEOF);
+		for (i = 0; i < dset->key.num_typeof; i++)
+			__nftnl_set_desc_build_typeof(udbuf, dset->data.expr[i], i);
+
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_UNSPEC:
+		return -1;
+	}
+
+	return 0;
+}
+
+EXPORT_SYMBOL(nftnl_set_desc_build_udata);
+int nftnl_set_desc_build_udata(struct nftnl_udata_buf *udbuf,
+			       const struct nftnl_set_desc *dset)
+{
+	if (!nftnl_udata_put_u32(udbuf, NFTNL_UDATA_SET_FLAGS, dset->flags))
+		return -1;
+
+	switch (dset->type) {
+	case NFTNL_DESC_SET_DATATYPE:
+		return nftnl_set_desc_build_dtype(udbuf, dset);
+	case NFTNL_DESC_SET_TYPEOF:

src/desc.c int nftnl_dtype_desc_parse(const struct nftnl_udata *attr, · line 442

Quoted 10 lines
src/desc.c +10 -0
+		return nftnl_set_desc_build_typeof(udbuf, dset);
+	case NFTNL_DESC_SET_UNSPEC:
+		return -1;
+	}
+
+	if (!nftnl_udata_put_strz(udbuf, NFTNL_UDATA_SET_COMMENT, dset->comment))
+		return -1;
+
+	return -1;
+}

This is odd: Depending on dset->type, nftnl_set_desc_build_udata() calls either nftnl_set_desc_build_dtype() or nftnl_set_desc_build_typeof(). Yet both check dset->type again. This looks like a mix-up of set/map key and data definitions and typeof vs. "regular" definition styles.

Cheers, Phil

reviewer
#7 lore raw
In reply to Pablo Neira Ayuso

Hi Pablo,

Quoted 4 lines · Pablo Neira Ayuso

This is my proposal to address the snprintf data printing depending on the arch. The idea is to add description objects that can be used to build the userdata area as well as to parse the userdata to create the description object.

I tried to integrate this into nftables, but failed to understand how this all is supposed to come together: In nftables, concat is treated like any other expression. Your series seems to require special treatment? At least there are separate "desc" data structures for each. It seems like one can't just replace build_udata callbacks to populate an nftnl_expr_desc object?

Cheers, Phil

author maintainer
#5 lore raw

Re: [PATCH libnftnl 3/3] desc: add set description

In reply to Phil Sutter
Quoted 99 lines · Phil Sutter

Hi,

On Thu, Jan 20, 2022 at 01:04:02AM +0100, Pablo Neira Ayuso wrote:
[...]

Quoted 4 lines
src/desc.c +0 -0
index f73e74c2c7d3..c8b3195db850 100644

[...]

Quoted 90 lines
src/desc.c +90 -0
+static int nftnl_set_desc_build_dtype(struct nftnl_udata_buf *udbuf,
+				      const struct nftnl_set_desc *dset)
+{
+	struct nftnl_udata *nest;
+	int i, err;
+
+	switch (dset->type) {
+	case NFTNL_DESC_SET_TYPEOF:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_KEY);
+		for (i = 0; i < dset->key.num_type; i++) {
+			err = __nftnl_udata_set_dtype_build(udbuf, dset->key.dtype[i], i);
+			if (err < 0)
+				return err;
+		}
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_DATATYPE:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_DATA);
+		for (i = 0; i < dset->data.num_type; i++) {
+			err = __nftnl_udata_set_dtype_build(udbuf, dset->data.dtype[i], i);
+			if (err < 0)
+				return err;
+		}
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_UNSPEC:
+		return -1;
+	}
+
+	return 0;
+}
+
+static int __nftnl_set_desc_build_typeof(struct nftnl_udata_buf *udbuf,
+					 const struct nftnl_expr_desc *dexpr,
+					 uint8_t attr_type)
+{
+	struct nftnl_udata *nest;
+	int err;
+
+	nest = nftnl_udata_nest_start(udbuf, attr_type);
+	err = nftnl_expr_desc_build(udbuf, dexpr);
+	nftnl_udata_nest_end(udbuf, nest);
+
+	return err;
+}
+
+static int nftnl_set_desc_build_typeof(struct nftnl_udata_buf *udbuf,
+				       const struct nftnl_set_desc *dset)
+{
+	struct nftnl_udata *nest;
+	int i;
+
+	switch (dset->type) {
+	case NFTNL_DESC_SET_TYPEOF:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_KEY_TYPEOF);
+		for (i = 0; i < dset->key.num_typeof; i++)
+			__nftnl_set_desc_build_typeof(udbuf, dset->key.expr[i], i);
+
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_DATATYPE:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_DATA_TYPEOF);
+		for (i = 0; i < dset->key.num_typeof; i++)
+			__nftnl_set_desc_build_typeof(udbuf, dset->data.expr[i], i);
+
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_UNSPEC:
+		return -1;
+	}
+
+	return 0;
+}
+
+EXPORT_SYMBOL(nftnl_set_desc_build_udata);
+int nftnl_set_desc_build_udata(struct nftnl_udata_buf *udbuf,
+			       const struct nftnl_set_desc *dset)
+{
+	if (!nftnl_udata_put_u32(udbuf, NFTNL_UDATA_SET_FLAGS, dset->flags))
+		return -1;
+
+	switch (dset->type) {
+	case NFTNL_DESC_SET_DATATYPE:
+		return nftnl_set_desc_build_dtype(udbuf, dset);
+	case NFTNL_DESC_SET_TYPEOF:
+		return nftnl_set_desc_build_typeof(udbuf, dset);
+	case NFTNL_DESC_SET_UNSPEC:
+		return -1;
+	}
+
Quoted 8 lines

src/desc.c int nftnl_dtype_desc_parse(const struct nftnl_udata *attr, · line 442

Quoted 5 lines
src/desc.c +5 -0
+	if (!nftnl_udata_put_strz(udbuf, NFTNL_UDATA_SET_COMMENT, dset->comment))
+		return -1;
+
+	return -1;
+}

This is odd: Depending on dset->type, nftnl_set_desc_build_udata() calls either nftnl_set_desc_build_dtype() or nftnl_set_desc_build_typeof().

That's indeed incorrect, nftnl_set_desc_build_typeof() should be:

static int nftnl_set_desc_build_typeof(struct nftnl_udata_buf *udbuf,
const struct nftnl_set_desc *dset)

{
	struct nftnl_udata *nest;
	int i;

	nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_KEY_TYPEOF);
	for (i = 0; i < dset->key.num_typeof; i++)
		__nftnl_set_desc_build_typeof(udbuf, dset->key.expr[i], i);

	nftnl_udata_nest_end(udbuf, nest);

	return 0;
}

The idea is: A set can either use datatype or typeof to define the elements that it stores. Then, a concatenation is possible.

Quoted 2 lines

Yet both check dset->type again. This looks like a mix-up of set/map key and data definitions and typeof vs. "regular" definition styles.

This patchset was a sketch PoC, I should have label it more explicit as such.

author maintainer
#8 lore raw
In reply to Phil Sutter
Quoted 8 lines · Phil Sutter

Hi Pablo,

Quoted 4 lines · Pablo Neira Ayuso

This is my proposal to address the snprintf data printing depending on the arch. The idea is to add description objects that can be used to build the userdata area as well as to parse the userdata to create the description object.

Quoted 4 lines

I tried to integrate this into nftables, but failed to understand how this all is supposed to come together: In nftables, concat is treated like any other expression. Your series seems to require special treatment?

The idea is that you build the nftnl description object either from the set typeof expression or the set datatype (depending on how the user has defined the set).

Quoted 3 lines

At least there are separate "desc" data structures for each. It seems like one can't just replace build_udata callbacks to populate an nftnl_expr_desc object?

You can use the description object in two ways:

- build_udata is called when setting the libnftnl set udata field, to build it.

- you pass the description object to snprintf.

The existing code to build the userdata TLV that resides in nftables should go away and use this new infrastructure, I'm basically moving to libnftnl the existing nftables code to build the set userdata area.

reviewer
#6 lore raw

Re: [PATCH libnftnl 3/3] desc: add set description

In reply to Pablo Neira Ayuso
Quoted 113 lines · Pablo Neira Ayuso
Quoted 99 lines · Phil Sutter

Hi,

On Thu, Jan 20, 2022 at 01:04:02AM +0100, Pablo Neira Ayuso wrote:
[...]

Quoted 4 lines
src/desc.c +0 -0
index f73e74c2c7d3..c8b3195db850 100644

[...]

Quoted 90 lines
src/desc.c +90 -0
+static int nftnl_set_desc_build_dtype(struct nftnl_udata_buf *udbuf,
+				      const struct nftnl_set_desc *dset)
+{
+	struct nftnl_udata *nest;
+	int i, err;
+
+	switch (dset->type) {
+	case NFTNL_DESC_SET_TYPEOF:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_KEY);
+		for (i = 0; i < dset->key.num_type; i++) {
+			err = __nftnl_udata_set_dtype_build(udbuf, dset->key.dtype[i], i);
+			if (err < 0)
+				return err;
+		}
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_DATATYPE:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_DATA);
+		for (i = 0; i < dset->data.num_type; i++) {
+			err = __nftnl_udata_set_dtype_build(udbuf, dset->data.dtype[i], i);
+			if (err < 0)
+				return err;
+		}
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_UNSPEC:
+		return -1;
+	}
+
+	return 0;
+}
+
+static int __nftnl_set_desc_build_typeof(struct nftnl_udata_buf *udbuf,
+					 const struct nftnl_expr_desc *dexpr,
+					 uint8_t attr_type)
+{
+	struct nftnl_udata *nest;
+	int err;
+
+	nest = nftnl_udata_nest_start(udbuf, attr_type);
+	err = nftnl_expr_desc_build(udbuf, dexpr);
+	nftnl_udata_nest_end(udbuf, nest);
+
+	return err;
+}
+
+static int nftnl_set_desc_build_typeof(struct nftnl_udata_buf *udbuf,
+				       const struct nftnl_set_desc *dset)
+{
+	struct nftnl_udata *nest;
+	int i;
+
+	switch (dset->type) {
+	case NFTNL_DESC_SET_TYPEOF:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_KEY_TYPEOF);
+		for (i = 0; i < dset->key.num_typeof; i++)
+			__nftnl_set_desc_build_typeof(udbuf, dset->key.expr[i], i);
+
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_DATATYPE:
+		nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_DATA_TYPEOF);
+		for (i = 0; i < dset->key.num_typeof; i++)
+			__nftnl_set_desc_build_typeof(udbuf, dset->data.expr[i], i);
+
+		nftnl_udata_nest_end(udbuf, nest);
+		break;
+	case NFTNL_DESC_SET_UNSPEC:
+		return -1;
+	}
+
+	return 0;
+}
+
+EXPORT_SYMBOL(nftnl_set_desc_build_udata);
+int nftnl_set_desc_build_udata(struct nftnl_udata_buf *udbuf,
+			       const struct nftnl_set_desc *dset)
+{
+	if (!nftnl_udata_put_u32(udbuf, NFTNL_UDATA_SET_FLAGS, dset->flags))
+		return -1;
+
+	switch (dset->type) {
+	case NFTNL_DESC_SET_DATATYPE:
+		return nftnl_set_desc_build_dtype(udbuf, dset);
+	case NFTNL_DESC_SET_TYPEOF:
+		return nftnl_set_desc_build_typeof(udbuf, dset);
+	case NFTNL_DESC_SET_UNSPEC:
+		return -1;
+	}
+
Quoted 8 lines

src/desc.c int nftnl_dtype_desc_parse(const struct nftnl_udata *attr, · line 442

Quoted 5 lines
src/desc.c +5 -0
+	if (!nftnl_udata_put_strz(udbuf, NFTNL_UDATA_SET_COMMENT, dset->comment))
+		return -1;
+
+	return -1;
+}

This is odd: Depending on dset->type, nftnl_set_desc_build_udata() calls either nftnl_set_desc_build_dtype() or nftnl_set_desc_build_typeof().

That's indeed incorrect, nftnl_set_desc_build_typeof() should be:

static int nftnl_set_desc_build_typeof(struct nftnl_udata_buf *udbuf,
const struct nftnl_set_desc *dset)

Quoted 12 lines
{
	struct nftnl_udata *nest;
	int i;

	nest = nftnl_udata_nest_start(udbuf, NFTNL_UDATA_SET_KEY_TYPEOF);
	for (i = 0; i < dset->key.num_typeof; i++)
		__nftnl_set_desc_build_typeof(udbuf, dset->key.expr[i], i);

	nftnl_udata_nest_end(udbuf, nest);

	return 0;
}

Ah, so only the NFTNL_DESC_SET_TYPEOF case content.

Quoted 8 lines

The idea is: A set can either use datatype or typeof to define the elements that it stores. Then, a concatenation is possible.

Quoted 2 lines

Yet both check dset->type again. This looks like a mix-up of set/map key and data definitions and typeof vs. "regular" definition styles.

This patchset was a sketch PoC, I should have label it more explicit as such.

Sure, I was aware of that. Figuring out how to make use of it is a bit of forensics though. The API is not identical to the one internally in nftables it replaces and since I don't know how it is supposed to function but only see how it may function based on the existing code, it is not easy to figure out which pieces are missing or where to adjust either this series or nftables code.

Cheers, Phil

reviewer
#9 lore raw
In reply to Pablo Neira Ayuso

Hi Pablo,

Quoted 8 lines · Pablo Neira Ayuso

This is my proposal to address the snprintf data printing depending on the arch. The idea is to add description objects that can be used to build the userdata area as well as to parse the userdata to create the description object.

This is revisiting 6e48df5329ea ("src: add "typeof" build/parse/print support") in nftables which adds build and parse userdata callbacks to expression in libnftables. My proposal is to move this to libnftnl.

Looking at your PoC again, I assume it was meant for use by applications to create and populate an nftnl_set_desc object and serialize it into nftnl_set's userdata using nftnl_set_desc_build_udata(). Since the information is needed within libnftnl though, the whole API does not make sense anymore and nftnl_set_desc must be serialized by libnftnl itself. This in turn means one may just integrate the data structure into nftnl_set's 'desc' field directly and extend nftnl_set_set_data() to allow populating the new fields, plus nftnl_set_desc_add_{expr,datatype}() I guess.

Am I on the right track there?

Maybe it's quicker for me to add the missing bits to my stuff instead of adjusting it to your series after making it work for the intended purpose. Especially since I'm not quite sure what goal we're trying to achieve.

Cheers, Phil

author maintainer
#10 lore raw
Quoted 22 lines · Phil Sutter

Hi Pablo,

Quoted 8 lines · Pablo Neira Ayuso

This is my proposal to address the snprintf data printing depending on the arch. The idea is to add description objects that can be used to build the userdata area as well as to parse the userdata to create the description object.

This is revisiting 6e48df5329ea ("src: add "typeof" build/parse/print support") in nftables which adds build and parse userdata callbacks to expression in libnftables. My proposal is to move this to libnftnl.

Looking at your PoC again, I assume it was meant for use by applications to create and populate an nftnl_set_desc object and serialize it into nftnl_set's userdata using nftnl_set_desc_build_udata(). Since the information is needed within libnftnl though, the whole API does not make sense anymore and nftnl_set_desc must be serialized by libnftnl itself. This in turn means one may just integrate the data structure into nftnl_set's 'desc' field directly and extend nftnl_set_set_data() to allow populating the new fields, plus nftnl_set_desc_add_{expr,datatype}() I guess.

Quoted 6 lines

Am I on the right track there?

Maybe it's quicker for me to add the missing bits to my stuff instead of adjusting it to your series after making it work for the intended purpose. Especially since I'm not quite sure what goal we're trying to achieve.

Goal is to consolidate code. Move the existing code in nftables to libnftnl so there is a desc object that can be use to build the userdata and to all assist the libnftnl print functions.

This will take a bit of work.