Skip to contentSkip to conversation

Support wildcard netdev hooks and events

PATCH Superseded Phil Sutter 3 participants 14 messages Oct 2, 2024 – Nov 6, 2024 netfilter

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Phil Sutter

author reviewer
#0 lore raw

cover Support wildcard netdev hooks and events

This series is the second (and last?) step of enabling support for name-based and wildcard interface hooks in user space. It depends on the previously sent series for libnftnl.

Patches 1-4 are fallout, fixing for deficits in different areas.

Patches 5 and 6 extend parser and serializer to accept and correctly pass interface wildcards on to the kernel.

Patch 7 adjusts shell test cases to the different behaviour (removed interfaces no longer disappearing from hook specs), mostly stored dump adjustments.

Patches 8 and 9 extend nft monitor to print NEWDEV/DELDEV events and extend the monitor testsuite to cover the code.

Phil Sutter (9):
json: Support typeof in set and map types
tests: py: Fix for storing payload into missing file
monitor: Recognize flowtable add/del events
tests: monitor: Run in own netns
mnl: Support simple wildcards in netdev hooks
parser_bison: Accept ASTERISK_STRING in flowtable_expr_member
tests: shell: Adjust to ifname-based flowtables
tests: monitor: Support running external commands
monitor: Support NFT_MSG_(NEW|DEL)DEV events

 doc/libnftables-json.adoc                     |   7 +-
 include/json.h                                |  20 +++
 include/linux/netfilter/nf_tables.h           |  10 ++
 include/netlink.h                             |   1 +
 include/rule.h                                |   1 +
 src/json.c                                    |  46 ++++++-
 src/mnl.c                                     |  19 ++-
 src/monitor.c                                 | 125 ++++++++++++++++++
 src/parser_bison.y                            |  11 +-
 src/parser_json.c                             |  15 +++
 src/rule.c                                    |  15 +++
 tests/monitor/run-tests.sh                    |  72 +++++++++-
 tests/monitor/testcases/chain-netdev.t        |  66 +++++++++
 tests/monitor/testcases/flowtable-simple.t    |  66 +++++++++
 tests/monitor/testcases/map-expr.t            |   2 +-
 tests/monitor/testcases/set-concat-interval.t |   3 +
 tests/py/nft-test.py                          |   5 +-
 .../chains/dumps/netdev_chain_0.json-nft      |  17 +++
 .../testcases/chains/dumps/netdev_chain_0.nft |   3 +
 .../netdev_chain_dormant_autoremove.json-nft  |   5 +-
 .../dumps/netdev_chain_dormant_autoremove.nft |   2 +-
 .../dumps/0012flowtable_variable_0.json-nft   |  10 +-
 .../dumps/0012flowtable_variable_0.nft        |   4 +-
 .../testcases/json/dumps/netdev.json-nft      |  13 ++
 tests/shell/testcases/json/dumps/netdev.nft   |   3 +
 .../listing/dumps/0020flowtable_0.json-nft    |   6 +-
 .../listing/dumps/0020flowtable_0.nft         |   2 +
 .../maps/dumps/0012map_concat_0.json-nft      |  21 ++-
 .../maps/dumps/0017_map_variable_0.json-nft   |  18 ++-
 .../maps/dumps/named_limits.json-nft          |  55 ++++++--
 .../dumps/typeof_maps_add_delete.json-nft     |   9 +-
 .../maps/dumps/typeof_maps_update_0.json-nft  |   9 +-
 .../maps/dumps/vmap_timeout.json-nft          |  22 ++-
 .../packetpath/dumps/set_lookups.json-nft     |  42 ++++--
 .../sets/dumps/0048set_counters_0.json-nft    |   9 +-
 .../testcases/sets/dumps/inner_0.json-nft     |  34 ++++-
 .../set_element_timeout_updates.json-nft      |   9 +-
 tests/shell/testcases/transactions/0050rule_1 |  19 ---
 .../transactions/dumps/0050rule_1.json-nft    |  11 --
 .../transactions/dumps/0050rule_1.nft         |   0
 40 files changed, 706 insertions(+), 101 deletions(-)
 create mode 100644 tests/monitor/testcases/chain-netdev.t
 create mode 100644 tests/monitor/testcases/flowtable-simple.t
 delete mode 100755 tests/shell/testcases/transactions/0050rule_1
 delete mode 100644 tests/shell/testcases/transactions/dumps/0050rule_1.json-nft
 delete mode 100644 tests/shell/testcases/transactions/dumps/0050rule_1.nft
Signature · 1 line
2.43.0
author reviewer
#1 lore raw

PATCH 1/9 json: Support typeof in set and map types

Implement this as a special "type" property value which is an object with sole property "typeof". The latter's value is the JSON representation of the expression in set->key, so for concatenated typeofs it is a concat expression.

All this is a bit clumsy right now but it works and it should be possible to tear it down a bit for more user-friendliness in a compatible way by either replacing the concat expression by the array it contains or even the whole "typeof" object - the parser would just assume any object (or objects in an array) in the "type" property value are expressions to extract a type from.

---
 doc/libnftables-json.adoc                     |  7 ++-
 src/json.c                                    | 13 ++++-
 src/parser_json.c                             |  9 +++
 tests/monitor/testcases/map-expr.t            |  2 +-
 tests/monitor/testcases/set-concat-interval.t |  3 +
 .../maps/dumps/0012map_concat_0.json-nft      | 21 +++++--
 .../maps/dumps/0017_map_variable_0.json-nft   | 18 +++++-
 .../maps/dumps/named_limits.json-nft          | 55 ++++++++++++++++---
 .../dumps/typeof_maps_add_delete.json-nft     |  9 ++-
 .../maps/dumps/typeof_maps_update_0.json-nft  |  9 ++-
 .../maps/dumps/vmap_timeout.json-nft          | 22 ++++++--
 .../packetpath/dumps/set_lookups.json-nft     | 42 +++++++++++---
 .../sets/dumps/0048set_counters_0.json-nft    |  9 ++-
 .../testcases/sets/dumps/inner_0.json-nft     | 34 ++++++++++--
 .../set_element_timeout_updates.json-nft      |  9 ++-
 15 files changed, 222 insertions(+), 40 deletions(-)
doc/libnftables-json.adoc +4 -3
index a8a6165fde59d..593d407c924e9 100644
@@ -341,7 +341,7 @@ ____
 	"auto-merge":* 'BOOLEAN'
 *}}*
 
-'SET_TYPE' := 'STRING' | *[* 'SET_TYPE_LIST' *]*
+'SET_TYPE' := 'STRING' | *[* 'SET_TYPE_LIST' *]* | *{ "typeof":* 'EXPRESSION' *}*
 'SET_TYPE_LIST' := 'STRING' [*,* 'SET_TYPE_LIST' ]
 'SET_POLICY' := *"performance"* | *"memory"*
 'SET_FLAG_LIST' := 'SET_FLAG' [*,* 'SET_FLAG_LIST' ]
@@ -381,8 +381,9 @@ that they translate a unique key to a value.
 	Automatic merging of adjacent/overlapping set elements in interval sets.
 
 ==== TYPE
-The set type might be a string, such as *"ipv4_addr"* or an array
-consisting of strings (for concatenated types).
+The set type might be a string, such as *"ipv4_addr"*, an array
+consisting of strings (for concatenated types) or a *typeof* object containing
+an expression to extract the type from.
 
 ==== ELEM
 A single set element might be given as string, integer or boolean value for
src/json.c +12 -1
index b1531ff3f4c9e..1f609bf2b03e9 100644
@@ -96,6 +96,17 @@ static json_t *set_dtype_json(const struct expr *key)
 	return root;
 }
 
+static json_t *set_key_dtype_json(const struct set *set,
+				  struct output_ctx *octx)
+{
+	bool use_typeof = set->key_typeof_valid;
+
+	if (!use_typeof)
+		return set_dtype_json(set->key);
+
+	return json_pack("{s:o}", "typeof", expr_print_json(set->key, octx));
+}
+
 static json_t *stmt_print_json(const struct stmt *stmt, struct output_ctx *octx)
 {
 	char buf[1024];
@@ -158,7 +169,7 @@ static json_t *set_print_json(struct output_ctx *octx, const struct set *set)
 			"family", family2str(set->handle.family),
 			"name", set->handle.set.name,
 			"table", set->handle.table.name,
-			"type", set_dtype_json(set->key),
+			"type", set_key_dtype_json(set, octx),
 			"handle", set->handle.handle.id);
 
 	if (set->comment)
src/parser_json.c +9 -0
index bbe3b1c59192c..f8200db1fe114 100644
@@ -1729,7 +1729,16 @@ static struct expr *json_parse_dtype_expr(struct json_ctx *ctx, json_t *root)
 			compound_expr_add(expr, i);
 		}
 		return expr;
+	} else if (json_is_object(root)) {
+		const char *key;
+		json_t *val;
+
+		if (!json_unpack_stmt(ctx, root, &key, &val) &&
+		    !strcmp(key, "typeof")) {
+			return json_parse_expr(ctx, val);
+		}
 	}
+
 	json_error(ctx, "Invalid set datatype.");
 	return NULL;
 }
tests/monitor/testcases/map-expr.t +1 -1
index 8729c0b44ee2c..d11ad0ebc0d57 100644
@@ -3,4 +3,4 @@ I add table ip t
 I add map ip t m { typeof meta day . meta hour : verdict; flags interval; counter; }
 O -
 J {"add": {"table": {"family": "ip", "name": "t", "handle": 0}}}
-J {"add": {"map": {"family": "ip", "name": "m", "table": "t", "type": ["day", "hour"], "handle": 0, "map": "verdict", "flags": ["interval"], "stmt": [{"counter": null}]}}}
+J {"add": {"map": {"family": "ip", "name": "m", "table": "t", "type": {"typeof": {"concat": [{"meta": {"key": "day"}}, {"meta": {"key": "hour"}}]}}, "handle": 0, "map": "verdict", "flags": ["interval"], "stmt": [{"counter": null}]}}}
tests/monitor/testcases/set-concat-interval.t +3 -0
index 763dc319f0d13..3542b8225ebd1 100644
@@ -10,3 +10,6 @@ I add map ip t s { typeof udp length . @ih,32,32 : verdict; flags interval; elem
 O add map ip t s { typeof udp length . @ih,32,32 : verdict; flags interval; }
 O add element ip t s { 20-80 . 0x14 : accept }
 O add element ip t s { 1-10 . 0xa : drop }
+J {"add": {"map": {"family": "ip", "name": "s", "table": "t", "type": {"typeof": {"concat": [{"payload": {"protocol": "udp", "field": "length"}}, {"payload": {"base": "ih", "offset": 32, "len": 32}}]}}, "handle": 0, "map": "verdict", "flags": ["interval"]}}}
+J {"add": {"element": {"family": "ip", "table": "t", "name": "s", "elem": {"set": [[{"concat": [{"range": [20, 80]}, 20]}, {"accept": null}]]}}}}
+J {"add": {"element": {"family": "ip", "table": "t", "name": "s", "elem": {"set": [[{"concat": [{"range": [1, 10]}, 10]}, {"drop": null}]]}}}}
tests/shell/testcases/maps/dumps/0012map_concat_0.json-nft +17 -4
index 000522365df9f..88bf4984dbde7 100644
@@ -31,10 +31,23 @@
         "family": "ip",
         "name": "w",
         "table": "x",
-        "type": [
-          "ipv4_addr",
-          "mark"
-        ],
+        "type": {
+          "typeof": {
+            "concat": [
+              {
+                "payload": {
+                  "protocol": "ip",
+                  "field": "saddr"
+                }
+              },
+              {
+                "meta": {
+                  "key": "mark"
+                }
+              }
+            ]
+          }
+        },
         "handle": 0,
         "map": "verdict",
         "flags": [
tests/shell/testcases/maps/dumps/0017_map_variable_0.json-nft +16 -2
index 725498cdcbef8..8eacf612d12fb 100644
@@ -19,7 +19,14 @@
         "family": "ip",
         "name": "y",
         "table": "x",
-        "type": "ipv4_addr",
+        "type": {
+          "typeof": {
+            "payload": {
+              "protocol": "ip",
+              "field": "saddr"
+            }
+          }
+        },
         "handle": 0,
         "map": "mark",
         "elem": [
@@ -39,7 +46,14 @@
         "family": "ip",
         "name": "z",
         "table": "x",
-        "type": "ipv4_addr",
+        "type": {
+          "typeof": {
+            "payload": {
+              "protocol": "ip",
+              "field": "saddr"
+            }
+          }
+        },
         "handle": 0,
         "map": "mark",
         "elem": [
tests/shell/testcases/maps/dumps/named_limits.json-nft +47 -8

79 lines not on this page · raw message

tests/shell/testcases/maps/dumps/typeof_maps_add_delete.json-nft +8 -1
index b3204a283d0ad..effe02dcf8364 100644
@@ -39,7 +39,14 @@
         "family": "ip",
         "name": "dynmark",
         "table": "dynset",
-        "type": "ipv4_addr",
+        "type": {
+          "typeof": {
+            "payload": {
+              "protocol": "ip",
+              "field": "daddr"
+            }
+          }
+        },
         "handle": 0,
         "map": "mark",
         "size": 64,
tests/shell/testcases/maps/dumps/typeof_maps_update_0.json-nft +8 -1
index 1d50477d783df..731514663b1aa 100644
@@ -50,7 +50,14 @@
         "family": "ip",
         "name": "sticky-set-svc-153CN2XYVUHRQ7UB",
         "table": "kube-nfproxy-v4",
-        "type": "ipv4_addr",
+        "type": {
+          "typeof": {
+            "payload": {
+              "protocol": "ip",
+              "field": "daddr"
+            }
+          }
+        },
         "handle": 0,
         "map": "mark",
         "size": 65535,
tests/shell/testcases/maps/dumps/vmap_timeout.json-nft +18 -4
index 1c3aa590f846e..71e9a9ee9f21b 100644
@@ -87,10 +87,24 @@
         "family": "inet",
         "name": "portaddrmap",
         "table": "filter",
-        "type": [
-          "ipv4_addr",
-          "inet_service"
-        ],
+        "type": {
+          "typeof": {
+            "concat": [
+              {
+                "payload": {
+                  "protocol": "ip",
+                  "field": "daddr"
+                }
+              },
+              {
+                "payload": {
+                  "protocol": "th",
+                  "field": "dport"
+                }
+              }
+            ]
+          }
+        },
         "handle": 0,
         "map": "verdict",
         "flags": [
tests/shell/testcases/packetpath/dumps/set_lookups.json-nft +34 -8

54 lines not on this page · raw message

tests/shell/testcases/sets/dumps/0048set_counters_0.json-nft +8 -1
index 62a6a177b7776..4be4112bf7935 100644
@@ -31,7 +31,14 @@
         "family": "ip",
         "name": "y",
         "table": "x",
-        "type": "ipv4_addr",
+        "type": {
+          "typeof": {
+            "payload": {
+              "protocol": "ip",
+              "field": "saddr"
+            }
+          }
+        },
         "handle": 0,
         "elem": [
           {
tests/shell/testcases/sets/dumps/inner_0.json-nft +29 -5
index 8d84e1ccecb9f..e5dc198f436be 100644
@@ -27,10 +27,26 @@
         "family": "netdev",
         "name": "x",
         "table": "x",
-        "type": [
-          "ipv4_addr",
-          "ipv4_addr"
-        ],
+        "type": {
+          "typeof": {
+            "concat": [
+              {
+                "payload": {
+                  "tunnel": "vxlan",
+                  "protocol": "ip",
+                  "field": "saddr"
+                }
+              },
+              {
+                "payload": {
+                  "tunnel": "vxlan",
+                  "protocol": "ip",
+                  "field": "daddr"
+                }
+              }
+            ]
+          }
+        },
         "handle": 0,
         "elem": [
           {
@@ -47,7 +63,15 @@
         "family": "netdev",
         "name": "y",
         "table": "x",
-        "type": "ipv4_addr",
+        "type": {
+          "typeof": {
+            "payload": {
+              "tunnel": "vxlan",
+              "protocol": "ip",
+              "field": "saddr"
+            }
+          }
+        },
         "handle": 0,
         "size": 65535,
         "flags": [
tests/shell/testcases/sets/dumps/set_element_timeout_updates.json-nft +8 -1
index aa908297e49ea..d92d8d7a54940 100644
@@ -31,7 +31,14 @@
         "family": "ip",
         "name": "s",
         "table": "t",
-        "type": "ipv4_addr",
+        "type": {
+          "typeof": {
+            "payload": {
+              "protocol": "ip",
+              "field": "saddr"
+            }
+          }
+        },
         "handle": 0,
         "flags": [
           "timeout"
Signature · 1 line
2.43.0
author reviewer
#2 lore raw

PATCH 2/9 tests: py: Fix for storing payload into missing file

In reply to Phil Sutter

When running a test for which no corresponding *.payload file exists, the *.payload.got file name was incorrectly constructed due to 'payload_path' variable not being set.

Fixes 2cfab7a3e10fc ("tests/py: Write dissenting payload into the right file")
---
 tests/py/nft-test.py | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
tests/py/nft-test.py +2 -3
index 00799e281d566..7acdb77f2d0a2 100755
@@ -769,10 +769,9 @@ def set_delete_elements(set_element, set_name, table, filename=None,
 
     if rule[1].strip() == "ok":
         payload_expected = None
-        payload_path = None
+        payload_path = "%s.payload" % filename_path
         try:
-            payload_log = open("%s.payload" % filename_path)
-            payload_path = payload_log.name
+            payload_log = open(payload_path)
             payload_expected = payload_find_expected(payload_log, rule[0])
         except:
             payload_log = None
Signature · 1 line
2.43.0
author reviewer
#3 lore raw

PATCH 3/9 monitor: Recognize flowtable add/del events

In reply to Phil Sutter

These were entirely ignored before, add the necessary code analogous to e.g. objects.

---
 include/json.h                             | 10 ++++
 include/netlink.h                          |  1 +
 include/rule.h                             |  1 +
 src/json.c                                 |  6 +++
 src/monitor.c                              | 61 ++++++++++++++++++++++
 src/parser_json.c                          |  6 +++
 src/rule.c                                 | 15 ++++++
 tests/monitor/testcases/flowtable-simple.t | 10 ++++
 8 files changed, 110 insertions(+)
 create mode 100644 tests/monitor/testcases/flowtable-simple.t
include/json.h +10 -0
index 39be8928e8ee0..0670b8714519b 100644
@@ -11,6 +11,7 @@ struct nlmsghdr;
 struct rule;
 struct set;
 struct obj;
+struct flowtable;
 struct stmt;
 struct symbol_table;
 struct table;
@@ -113,6 +114,8 @@ void monitor_print_element_json(struct netlink_mon_handler *monh,
 				const char *cmd, struct set *s);
 void monitor_print_obj_json(struct netlink_mon_handler *monh,
 			    const char *cmd, struct obj *o);
+void monitor_print_flowtable_json(struct netlink_mon_handler *monh,
+				  const char *cmd, struct flowtable *ft);
 void monitor_print_rule_json(struct netlink_mon_handler *monh,
 			     const char *cmd, struct rule *r);
 
@@ -254,6 +257,13 @@ static inline void monitor_print_obj_json(struct netlink_mon_handler *monh,
 	/* empty */
 }
 
+static inline void
+monitor_print_flowtable_json(struct netlink_mon_handler *monh,
+			     const char *cmd, struct flowtable *ft)
+{
+	/* empty */
+}
+
 static inline void monitor_print_rule_json(struct netlink_mon_handler *monh,
 					   const char *cmd, struct rule *r)
 {
include/netlink.h +1 -0
index cf7ba3693885a..e9667a24b0d11 100644
@@ -97,6 +97,7 @@ extern struct nftnl_table *netlink_table_alloc(const struct nlmsghdr *nlh);
 extern struct nftnl_chain *netlink_chain_alloc(const struct nlmsghdr *nlh);
 extern struct nftnl_set *netlink_set_alloc(const struct nlmsghdr *nlh);
 extern struct nftnl_obj *netlink_obj_alloc(const struct nlmsghdr *nlh);
+extern struct nftnl_flowtable *netlink_flowtable_alloc(const struct nlmsghdr *nlh);
 extern struct nftnl_rule *netlink_rule_alloc(const struct nlmsghdr *nlh);
 
 struct nft_data_linearize {
include/rule.h +1 -0
index 5b3e12b5d7dcf..75166b48446f5 100644
@@ -551,6 +551,7 @@ extern struct flowtable *flowtable_lookup_fuzzy(const char *ft_name,
 						const struct table **table);
 
 void flowtable_print(const struct flowtable *n, struct output_ctx *octx);
+void flowtable_print_plain(const struct flowtable *ft, struct output_ctx *octx);
 
 /**
  * enum cmd_ops - command operations
src/json.c +6 -0
index 1f609bf2b03e9..64a6888f9e0ac 100644
@@ -2108,6 +2108,12 @@ void monitor_print_obj_json(struct netlink_mon_handler *monh,
 	monitor_print_json(monh, cmd, obj_print_json(o));
 }
 
+void monitor_print_flowtable_json(struct netlink_mon_handler *monh,
+				  const char *cmd, struct flowtable *ft)
+{
+	monitor_print_json(monh, cmd, flowtable_print_json(ft));
+}
+
 void monitor_print_rule_json(struct netlink_mon_handler *monh,
 			     const char *cmd, struct rule *r)
 {
src/monitor.c +61 -0

79 lines not on this page · raw message

src/parser_json.c +6 -0
index f8200db1fe114..bcc216e12e51c 100644
@@ -4421,6 +4421,7 @@ static int json_echo_error(struct netlink_mon_handler *monh,
 
 static uint64_t handle_from_nlmsg(const struct nlmsghdr *nlh)
 {
+	struct nftnl_flowtable *nlf;
 	struct nftnl_table *nlt;
 	struct nftnl_chain *nlc;
 	struct nftnl_rule *nlr;
@@ -4457,6 +4458,11 @@ static uint64_t handle_from_nlmsg(const struct nlmsghdr *nlh)
 		handle = nftnl_obj_get_u64(nlo, NFTNL_OBJ_HANDLE);
 		nftnl_obj_free(nlo);
 		break;
+	case NFT_MSG_NEWFLOWTABLE:
+		nlf = netlink_flowtable_alloc(nlh);
+		handle = nftnl_flowtable_get_u64(nlf, NFTNL_FLOWTABLE_HANDLE);
+		nftnl_flowtable_free(nlf);
+		break;
 	}
 	return handle;
 }
src/rule.c +15 -0
index 9bc160ec0d888..dc6b9d89fc967 100644
@@ -2155,6 +2155,21 @@ void flowtable_print(const struct flowtable *s, struct output_ctx *octx)
 	do_flowtable_print(s, &opts, octx);
 }
 
+void flowtable_print_plain(const struct flowtable *ft, struct output_ctx *octx)
+{
+	struct print_fmt_options opts = {
+		.tab		= "",
+		.nl		= " ",
+		.table		= ft->handle.table.name,
+		.family		= family2str(ft->handle.family),
+		.stmt_separator = "; ",
+	};
+
+	flowtable_print_declaration(ft, &opts, octx);
+	nft_print(octx, "}");
+}
+
+
 struct flowtable *flowtable_lookup_fuzzy(const char *ft_name,
 					 const struct nft_cache *cache,
 					 const struct table **t)
tests/monitor/testcases/flowtable-simple.t new file +10 -0
new file mode 100644
index 0000000000000..df8eccbd91e0a
@@ -0,0 +1,10 @@
+# setup first
+I add table ip t
+I add flowtable ip t ft { hook ingress priority 0; devices = { lo }; }
+O -
+J {"add": {"table": {"family": "ip", "name": "t", "handle": 0}}}
+J {"add": {"flowtable": {"family": "ip", "name": "ft", "table": "t", "handle": 0, "hook": "ingress", "prio": 0, "dev": "lo"}}}
+
+I delete flowtable ip t ft
+O -
+J {"delete": {"flowtable": {"family": "ip", "name": "ft", "table": "t", "handle": 0, "hook": "ingress", "prio": 0, "dev": "lo"}}}
Signature · 1 line
2.43.0
author reviewer
#4 lore raw

PATCH 4/9 tests: monitor: Run in own netns

In reply to Phil Sutter

Have the script call itself prefixed by unshare. This won't prevent clashing test case contents, but at least leave the host netns alone.

---
 tests/monitor/run-tests.sh | 11 +++++++++++
 1 file changed, 11 insertions(+)
tests/monitor/run-tests.sh +11 -0
index f1ac790acf80c..214512d269e8d 100755
@@ -120,6 +120,14 @@ echo_run_test() {
 	return $rc
 }
 
+netns=true
+for arg in "$@"; do
+	[[ "$arg" == "--no-netns" ]] && netns=false
+done
+if $netns; then
+	exec unshare -n $0 --no-netns "$@"
+fi
+
 testcases=""
 while [ -n "$1" ]; do
 	case "$1" in
@@ -131,6 +139,9 @@ while [ -n "$1" ]; do
 		test_json=true
 		shift
 		;;
+	--no-netns)
+		shift
+		;;
 	-H|--host)
 		nft=nft
 		shift
Signature · 1 line
2.43.0
author reviewer
#5 lore raw

PATCH 5/9 mnl: Support simple wildcards in netdev hooks

In reply to Phil Sutter

When building NFTA_FLOWTABLE_HOOK_DEVS, NFTA_HOOK_DEV or NFTA_HOOK_DEVS attributes, detect trailing asterisks in interface names and reduce attribute length accordingly. Kernel will use strncmp(), effectively performing a prefix search this way.

Deserialization (i.e., appending asterisk to interface names which don't include a trailing nul-char) happens in libnftnl.

---
 src/mnl.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)
src/mnl.c +14 -5
index db53a60b43cb9..4faf027ce1027 100644
@@ -787,18 +787,24 @@ static void mnl_nft_chain_devs_build(struct nlmsghdr *nlh, struct cmd *cmd)
 {
 	const struct expr *dev_expr = cmd->chain->dev_expr;
 	const struct nft_dev *dev_array;
+	int i, len, num_devs = 0;
 	struct nlattr *nest_dev;
-	int i, num_devs = 0;
 
 	dev_array = nft_dev_array(dev_expr, &num_devs);
 	if (num_devs == 1) {
 		cmd_add_loc(cmd, nlh->nlmsg_len, dev_array[0].location);
-		mnl_attr_put_strz(nlh, NFTA_HOOK_DEV, dev_array[0].ifname);
+		len = strlen(dev_array[0].ifname) + 1;
+		if (dev_array[0].ifname[len - 2] == '*')
+			len -= 2;
+		mnl_attr_put(nlh, NFTA_HOOK_DEV, len, dev_array[0].ifname);
 	} else {
 		nest_dev = mnl_attr_nest_start(nlh, NFTA_HOOK_DEVS);
 		for (i = 0; i < num_devs; i++) {
 			cmd_add_loc(cmd, nlh->nlmsg_len, dev_array[i].location);
-			mnl_attr_put_strz(nlh, NFTA_DEVICE_NAME, dev_array[i].ifname);
+			len = strlen(dev_array[i].ifname) + 1;
+			if (dev_array[i].ifname[len - 2] == '*')
+				len -= 2;
+			mnl_attr_put(nlh, NFTA_DEVICE_NAME, len, dev_array[i].ifname);
 			mnl_attr_nest_end(nlh, nest_dev);
 		}
 	}
@@ -1999,14 +2005,17 @@ static void mnl_nft_ft_devs_build(struct nlmsghdr *nlh, struct cmd *cmd)
 {
 	const struct expr *dev_expr = cmd->flowtable->dev_expr;
 	const struct nft_dev *dev_array;
+	int i, len, num_devs = 0;
 	struct nlattr *nest_dev;
-	int i, num_devs= 0;
 
 	dev_array = nft_dev_array(dev_expr, &num_devs);
 	nest_dev = mnl_attr_nest_start(nlh, NFTA_FLOWTABLE_HOOK_DEVS);
 	for (i = 0; i < num_devs; i++) {
 		cmd_add_loc(cmd, nlh->nlmsg_len, dev_array[i].location);
-		mnl_attr_put_strz(nlh, NFTA_DEVICE_NAME, dev_array[i].ifname);
+		len = strlen(dev_array[i].ifname) + 1;
+		if (dev_array[i].ifname[len - 2] == '*')
+			len -= 2;
+		mnl_attr_put(nlh, NFTA_DEVICE_NAME, len, dev_array[i].ifname);
 	}
 
 	mnl_attr_nest_end(nlh, nest_dev);
Signature · 1 line
2.43.0
author reviewer
#6 lore raw

PATCH 6/9 parser_bison: Accept ASTERISK_STRING in flowtable_expr_member

In reply to Phil Sutter

All clauses are identical, so instead of adding a third one for ASTERISK_STRING, use a single one for 'string' (which combines all three variants).

---
 src/parser_bison.y | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)
src/parser_bison.y +1 -10
index e2936d10efe4c..d9cf2cd25c2f0 100644
@@ -2477,16 +2477,7 @@ flowtable_list_expr	:	flowtable_expr_member
 			|	flowtable_list_expr	COMMA	opt_newline
 			;
 
-flowtable_expr_member	:	QUOTED_STRING
-			{
-				struct expr *expr = ifname_expr_alloc(&@$, state->msgs, $1);
-
-				if (!expr)
-					YYERROR;
-
-				$$ = expr;
-			}
-			|	STRING
+flowtable_expr_member	:	string
 			{
 				struct expr *expr = ifname_expr_alloc(&@$, state->msgs, $1);
 
Signature · 1 line
2.43.0
author reviewer
#7 lore raw

PATCH 7/9 tests: shell: Adjust to ifname-based flowtables

In reply to Phil Sutter

Removed interfaces will remain in place in dumps. Also drop transactions/0050rule_1 test entirely: It won't fail anymore as the flowtable is accepted despite the non-existent interfaces and thus the test as a whole does not work anymore.

---
 .../chains/dumps/netdev_chain_0.json-nft      | 17 +++++++++++++++++
 .../testcases/chains/dumps/netdev_chain_0.nft |  3 +++
 .../netdev_chain_dormant_autoremove.json-nft  |  5 ++++-
 .../dumps/netdev_chain_dormant_autoremove.nft |  2 +-
 .../dumps/0012flowtable_variable_0.json-nft   | 10 ++++++++--
 .../dumps/0012flowtable_variable_0.nft        |  4 ++--
 .../testcases/json/dumps/netdev.json-nft      | 13 +++++++++++++
 tests/shell/testcases/json/dumps/netdev.nft   |  3 +++
 .../listing/dumps/0020flowtable_0.json-nft    |  6 ++++--
 .../listing/dumps/0020flowtable_0.nft         |  2 ++
 tests/shell/testcases/transactions/0050rule_1 | 19 -------------------
 .../transactions/dumps/0050rule_1.json-nft    | 11 -----------
 .../transactions/dumps/0050rule_1.nft         |  0
 13 files changed, 57 insertions(+), 38 deletions(-)
 delete mode 100755 tests/shell/testcases/transactions/0050rule_1
 delete mode 100644 tests/shell/testcases/transactions/dumps/0050rule_1.json-nft
 delete mode 100644 tests/shell/testcases/transactions/dumps/0050rule_1.nft
tests/shell/testcases/chains/dumps/netdev_chain_0.json-nft +17 -0
index 7d78bd6757034..13e9f6bb016f7 100644
@@ -13,6 +13,23 @@
         "name": "x",
         "handle": 0
       }
+    },
+    {
+      "chain": {
+        "family": "netdev",
+        "table": "x",
+        "name": "y",
+        "handle": 0,
+        "dev": [
+          "d0",
+          "d1",
+          "d2"
+        ],
+        "type": "filter",
+        "hook": "ingress",
+        "prio": 0,
+        "policy": "accept"
+      }
     }
   ]
 }
tests/shell/testcases/chains/dumps/netdev_chain_0.nft +3 -0
index aa571e00885fe..6606d5bc3f608 100644
@@ -1,2 +1,5 @@
 table netdev x {
+	chain y {
+		type filter hook ingress devices = { d0, d1, d2 } priority filter; policy accept;
+	}
 }
tests/shell/testcases/chains/dumps/netdev_chain_dormant_autoremove.json-nft +4 -1
index 9151d42f17d91..88b8958f4d86e 100644
@@ -21,7 +21,10 @@
         "table": "test",
         "name": "ingress",
         "handle": 0,
-        "dev": "dummy1",
+        "dev": [
+          "dummy0",
+          "dummy1"
+        ],
         "type": "filter",
         "hook": "ingress",
         "prio": 0,
tests/shell/testcases/chains/dumps/netdev_chain_dormant_autoremove.nft +1 -1
index aad7cb6337734..f4bd9556b3e03 100644
@@ -2,6 +2,6 @@ table netdev test {
 	flags dormant
 
 	chain ingress {
-		type filter hook ingress device "dummy1" priority filter; policy drop;
+		type filter hook ingress devices = { dummy0, dummy1 } priority filter; policy drop;
 	}
 }
tests/shell/testcases/flowtable/dumps/0012flowtable_variable_0.json-nft +8 -2
index 10f1df98874ab..20da08fb2fc29 100644
@@ -22,7 +22,10 @@
         "handle": 0,
         "hook": "ingress",
         "prio": 0,
-        "dev": "lo"
+        "dev": [
+          "dummy1",
+          "lo"
+        ]
       }
     },
     {
@@ -40,7 +43,10 @@
         "handle": 0,
         "hook": "ingress",
         "prio": 0,
-        "dev": "lo"
+        "dev": [
+          "dummy1",
+          "lo"
+        ]
       }
     }
   ]
tests/shell/testcases/flowtable/dumps/0012flowtable_variable_0.nft +2 -2
index df1c51a247033..1cbb2f1103f03 100644
@@ -1,14 +1,14 @@
 table ip filter1 {
 	flowtable Main_ft1 {
 		hook ingress priority filter
-		devices = { lo }
+		devices = { dummy1, lo }
 		counter
 	}
 }
 table ip filter2 {
 	flowtable Main_ft2 {
 		hook ingress priority filter
-		devices = { lo }
+		devices = { dummy1, lo }
 		counter
 	}
 }
tests/shell/testcases/json/dumps/netdev.json-nft +13 -0
index e0d2bfb4385b7..6eb19a17b31d9 100644
@@ -13,6 +13,19 @@
         "name": "test_table",
         "handle": 0
       }
+    },
+    {
+      "chain": {
+        "family": "netdev",
+        "table": "test_table",
+        "name": "test_chain",
+        "handle": 0,
+        "dev": "d0",
+        "type": "filter",
+        "hook": "ingress",
+        "prio": 0,
+        "policy": "accept"
+      }
     }
   ]
 }
tests/shell/testcases/json/dumps/netdev.nft +3 -0
index 3c568ed3eb38d..373ea0a46d600 100644
@@ -1,2 +1,5 @@
 table netdev test_table {
+	chain test_chain {
+		type filter hook ingress device "d0" priority filter; policy accept;
+	}
 }
tests/shell/testcases/listing/dumps/0020flowtable_0.json-nft +4 -2
index d511739abd4b6..b1b3a5fba34a0 100644
@@ -32,7 +32,8 @@
         "table": "filter",
         "handle": 0,
         "hook": "ingress",
-        "prio": 0
+        "prio": 0,
+        "dev": "d0"
       }
     },
     {
@@ -60,7 +61,8 @@
         "table": "filter",
         "handle": 0,
         "hook": "ingress",
-        "prio": 0
+        "prio": 0,
+        "dev": "d0"
       }
     }
   ]
tests/shell/testcases/listing/dumps/0020flowtable_0.nft +2 -0
index 4a64e531db840..59fcbec8e5130 100644
@@ -6,6 +6,7 @@ table inet filter {
 
 	flowtable f2 {
 		hook ingress priority filter
+		devices = { d0 }
 	}
 }
 table ip filter {
@@ -16,5 +17,6 @@ table ip filter {
 
 	flowtable f2 {
 		hook ingress priority filter
+		devices = { d0 }
 	}
 }
tests/shell/testcases/transactions/0050rule_1 deleted +0 -19
deleted file mode 100755
index 89e5f42fc9f4d..0000000000000
@@ -1,19 +0,0 @@
-#!/bin/bash
-
-set -e
-
-RULESET="table inet filter {
-	flowtable ftable {
-		hook ingress priority 0; devices = { eno1, eno0, x };
-	}
-
-chain forward {
-	type filter hook forward priority 0; policy drop;
-
-	ip protocol { tcp, udp } ct mark and 1 == 1 counter flow add @ftable
-	ip6 nexthdr { tcp, udp } ct mark and 2 == 2 counter flow add @ftable
-	ct mark and 30 == 30 ct state established,related log prefix \"nftables accept: \" level info accept
-	}
-}"
-
-$NFT -f - <<< "$RULESET" >/dev/null || exit 0
tests/shell/testcases/transactions/dumps/0050rule_1.json-nft deleted +0 -11
deleted file mode 100644
index 546cc5977db61..0000000000000
@@ -1,11 +0,0 @@
-{
-  "nftables": [
-    {
-      "metainfo": {
-        "version": "VERSION",
-        "release_name": "RELEASE_NAME",
-        "json_schema_version": 1
-      }
-    }
-  ]
-}
tests/shell/testcases/transactions/dumps/0050rule_1.nft deleted +0 -0
deleted file mode 100644
index e69de29bb2d1d..0000000000000
Signature · 1 line
2.43.0
author reviewer
#8 lore raw

PATCH 8/9 tests: monitor: Support running external commands

In reply to Phil Sutter

Accept '@'-prefixed lines mixed in between 'I'-prefixed ones and apply nft input and run commands in specified ordering.

To keep things simple, ignore such test cases when in echo mode for now.

---
 tests/monitor/run-tests.sh | 61 ++++++++++++++++++++++++++++++++++----
 1 file changed, 56 insertions(+), 5 deletions(-)
tests/monitor/run-tests.sh +56 -5
index 214512d269e8d..36fc800c10520 100755
@@ -59,6 +59,44 @@ json_output_filter() { # (filename)
 	# unify handle values
 	sed -i -e 's/\("handle":\) [0-9][0-9]*/\1 0/g' "$1"
 }
+apply_commands_from_file() { # (command_file)
+	grep -q '^# run: ' $1 || {
+		$nft -f - <$1 || {
+			err "nft command failed!"
+			return 1
+		}
+		return 0
+	}
+	local nft_cmd=""
+	local sep=""
+	local line=""
+	while read line; do
+		[[ $line =~ ^#\ run: ]] || {
+			nft_cmd+="$sep$line"
+			sep="; "
+			continue
+		}
+		[[ -n $nft_cmd ]] && {
+			$nft -f - <<<"$nft_cmd" || {
+				err "nft command failed!"
+				return 1
+			}
+		}
+		nft_cmd=""
+		sep=""
+		${line#\# run: } || {
+			err "custom command failed!"
+			return 1
+		}
+	done <$1
+	[[ -n $nft_cmd ]] && {
+		$nft -f - <<<"$nft_cmd" || {
+			err "nft command failed!"
+			return 1
+		}
+	}
+	return 0
+}
 monitor_run_test() {
 	monitor_output=$(mktemp -p $testdir)
 	monitor_args=""
@@ -74,10 +112,7 @@ monitor_run_test() {
 		echo "command file:"
 		cat $command_file
 	}
-	$nft -f - <$command_file || {
-		err "nft command failed!"
-		rc=1
-	}
+	apply_commands_from_file $command_file || rc=1
 	sleep 0.5
 	kill $monitor_pid
 	wait >/dev/null 2>&1
@@ -103,6 +138,17 @@ echo_run_test() {
 		echo "command file:"
 		cat $command_file
 	}
+	grep -q '^# run: ' $command_file && {
+		$debug && {
+			echo "skipping unsuitable test case in command file:"
+			cat $command_file
+		}
+		rm $command_file
+		rm $output_file
+		touch $command_file
+		touch $output_file
+		return 0
+	}
 	$nft -nn -e -f - <$command_file >$echo_output || {
 		err "nft command failed!"
 		rc=1
@@ -182,18 +228,23 @@ for variant in $variants; do
 		# O add table ip t
 		# I add chain ip t c
 		# O add chain ip t c
+		# I <nft input1>
+		# @ <command to run after nft input1>
+		# I <nft input2>
+		# O ...
 
 		$nft flush ruleset
 
 		input_complete=false
 		while read dir line; do
 			case $dir in
-			I)
+			I|@)
 				$input_complete && {
 					$run_test
 					let "rc += $?"
 				}
 				input_complete=false
+				[[ $dir == '@' ]] && line="# run: $line"
 				cmd_append "$line"
 				;;
 			O)
Signature · 1 line
2.43.0
author reviewer
#9 lore raw

PATCH 9/9 monitor: Support NFT_MSG_(NEW|DEL)DEV events

In reply to Phil Sutter

Kernels with name-based netdev hooks emit these messages when a device is added to or removed from an existing flowtable or netdev-family base-chain.

This patch depends on respective support code in libnftnl.

---
 include/json.h                             | 10 ++++
 include/linux/netfilter/nf_tables.h        | 10 ++++
 src/json.c                                 | 27 +++++++++
 src/monitor.c                              | 64 +++++++++++++++++++++
 tests/monitor/testcases/chain-netdev.t     | 66 ++++++++++++++++++++++
 tests/monitor/testcases/flowtable-simple.t | 56 ++++++++++++++++++
 6 files changed, 233 insertions(+)
 create mode 100644 tests/monitor/testcases/chain-netdev.t
include/json.h +10 -0
index 0670b8714519b..10a75ba050a6c 100644
@@ -20,6 +20,7 @@ struct nft_ctx;
 struct location;
 struct output_ctx;
 struct list_head;
+struct nftnl_device;
 
 #ifdef HAVE_LIBJANSSON
 
@@ -118,6 +119,8 @@ void monitor_print_flowtable_json(struct netlink_mon_handler *monh,
 				  const char *cmd, struct flowtable *ft);
 void monitor_print_rule_json(struct netlink_mon_handler *monh,
 			     const char *cmd, struct rule *r);
+void monitor_print_device_json(struct netlink_mon_handler *monh,
+			       const char *cmd, struct nftnl_device *nld);
 
 int json_events_cb(const struct nlmsghdr *nlh,
 		   struct netlink_mon_handler *monh);
@@ -270,6 +273,13 @@ static inline void monitor_print_rule_json(struct netlink_mon_handler *monh,
 	/* empty */
 }
 
+static inline void
+monitor_print_device_json(struct netlink_mon_handler *monh,
+			  const char *cmd, struct nftnl_device *nld)
+{
+	/* empty */
+}
+
 static inline int json_events_cb(const struct nlmsghdr *nlh,
                                  struct netlink_mon_handler *monh)
 {
include/linux/netfilter/nf_tables.h +10 -0
index c62e6ac563988..206d90b190951 100644
@@ -142,6 +142,8 @@ enum nf_tables_msg_types {
 	NFT_MSG_DESTROYOBJ,
 	NFT_MSG_DESTROYFLOWTABLE,
 	NFT_MSG_GETSETELEM_RESET,
+	NFT_MSG_NEWDEV,
+	NFT_MSG_DELDEV,
 	NFT_MSG_MAX,
 };
 
@@ -1761,10 +1763,18 @@ enum nft_synproxy_attributes {
  * enum nft_device_attributes - nf_tables device netlink attributes
  *
  * @NFTA_DEVICE_NAME: name of this device (NLA_STRING)
+ * @NFTA_DEVICE_TABLE: table containing the flowtable or chain hooking into the device (NLA_STRING)
+ * @NFTA_DEVICE_FLOWTABLE: flowtable hooking into the device (NLA_STRING)
+ * @NFTA_DEVICE_CHAIN: chain hooking into the device (NLA_STRING)
+ * @NFTA_DEVICE_SPEC: hook spec matching the device (NLA_STRING)
  */
 enum nft_devices_attributes {
 	NFTA_DEVICE_UNSPEC,
 	NFTA_DEVICE_NAME,
+	NFTA_DEVICE_TABLE,
+	NFTA_DEVICE_FLOWTABLE,
+	NFTA_DEVICE_CHAIN,
+	NFTA_DEVICE_SPEC,
 	__NFTA_DEVICE_MAX
 };
 #define NFTA_DEVICE_MAX		(__NFTA_DEVICE_MAX - 1)
src/json.c +27 -0
index 64a6888f9e0ac..02f21bb6b0d92 100644
@@ -17,6 +17,8 @@
 #include <rt.h>
 #include "nftutils.h"
 
+#include <libnftnl/device.h>
+
 #include <netdb.h>
 #include <netinet/icmp6.h>
 #include <netinet/in.h>
@@ -2122,6 +2124,31 @@ void monitor_print_rule_json(struct netlink_mon_handler *monh,
 	monitor_print_json(monh, cmd, rule_print_json(octx, r));
 }
 
+void monitor_print_device_json(struct netlink_mon_handler *monh,
+			       const char *cmd, struct nftnl_device *nld)
+{
+	int32_t family = nftnl_device_get_s32(nld, NFTNL_DEVICE_FAMILY);
+	const char *key, *val;
+	json_t *root;
+
+	if (nftnl_device_is_set(nld, NFTNL_DEVICE_CHAIN)) {
+		key = "chain";
+		val = nftnl_device_get_str(nld, NFTNL_DEVICE_CHAIN);
+	} else if (nftnl_device_is_set(nld, NFTNL_DEVICE_FLOWTABLE)) {
+		key = "flowtable";
+		val = nftnl_device_get_str(nld, NFTNL_DEVICE_FLOWTABLE);
+	} else {
+		return;
+	}
+	root = json_pack("{s:{s:s, s:s, s:s, s:s, s:s}}", "device",
+			 "family", family2str(family),
+			 "table", nftnl_device_get_str(nld, NFTNL_DEVICE_TABLE),
+			 key, val,
+			 "name", nftnl_device_get_str(nld, NFTNL_DEVICE_NAME),
+			 "spec", nftnl_device_get_str(nld, NFTNL_DEVICE_SPEC));
+	monitor_print_json(monh, cmd, root);
+}
+
 void json_alloc_echo(struct nft_ctx *nft)
 {
 	nft->json_echo = json_array();
src/monitor.c +64 -0

100 lines not on this page · raw message

tests/monitor/testcases/chain-netdev.t new file +66 -0

66 lines not on this page · raw message

tests/monitor/testcases/flowtable-simple.t +56 -0

59 lines not on this page · raw message

Signature · 1 line
2.43.0
author reviewer
#10 lore raw

Re: [nft PATCH 9/9] monitor: Support NFT_MSG_(NEW|DEL)DEV events

On Wed, Oct 02, 2024 at 09:38:53PM +0200, Phil Sutter wrote:
[...]

Quoted 2 lines
 tests/monitor/testcases/chain-netdev.t     | 66 ++++++++++++++++++++++
 tests/monitor/testcases/flowtable-simple.t | 56 ++++++++++++++++++

And as foretold in patch 4, the new chain-netdev.t clashes with similar content in flowtable-simple.t. To avoid this, both need a trivial adjustment:

tests/monitor/testcases/chain-netdev.t +1 -0
index 3c004af0cd855..414040045dd20 100644
@@ -48,6 +48,7 @@ I add chain netdev t c2 { type filter hook ingress devices = { wald* } priority
 @ ip link del wild23
 I delete chain netdev t c
 I delete chain netdev t c2
+@ ip link del wald42
 O add chain netdev t c { type filter hook ingress devices = { wild* } priority 0; policy accept; }
 O add chain netdev t c2 { type filter hook ingress devices = { wald* } priority 0; policy accept; }
 O add device chain netdev t c hook wild* { wild23 }
tests/monitor/testcases/flowtable-simple.t +1 -0
index 113b15f20d1dc..2c82f4907ed93 100644
@@ -48,6 +48,7 @@ I add flowtable ip t ft2 { hook ingress priority 0; devices = { wald* }; }
 @ ip link del wild23
 I delete flowtable ip t ft
 I delete flowtable ip t ft2
+@ ip link del wald42
 O add flowtable ip t ft { hook ingress priority 0; devices = { wild* }; }
 O add flowtable ip t ft2 { hook ingress priority 0; devices = { wald* }; }
 O add device flowtable ip t ft hook wild* { wild23 }

I'll wait for feedback before including this in a v2. Sorry for the mess.

maintainer
#11 lore raw
In reply to Phil Sutter
Quoted 5 lines · Phil Sutter

This series is the second (and last?) step of enabling support for name-based and wildcard interface hooks in user space. It depends on the previously sent series for libnftnl.

Patches 1-4 are fallout, fixing for deficits in different areas.

These look good, happy to see typeof support on json side, feel free to push them out.

#12 lore raw
Quoted 9 lines · Florian Westphal
Quoted 5 lines · Phil Sutter

This series is the second (and last?) step of enabling support for name-based and wildcard interface hooks in user space. It depends on the previously sent series for libnftnl.

Patches 1-4 are fallout, fixing for deficits in different areas.

These look good, happy to see typeof support on json side, feel free to push them out.

Ack for 1-4 in this series, thanks.

author reviewer
#13 lore raw
In reply to Phil Sutter
Quoted 5 lines · Phil Sutter

This series is the second (and last?) step of enabling support for name-based and wildcard interface hooks in user space. It depends on the previously sent series for libnftnl.

Patches 1-4 are fallout, fixing for deficits in different areas.

Applied these four patches.