Skip to contentSkip to conversation

psp: Add support for dev-assoc/disassoc

PATCH Superseded Wei Wang 2 participants 12 messages Mar 22 – Mar 26

Subsystems (5)

v7 of 16 Superseded Latest is v16 →

lore
Share & tools
lore

Wei Wang

author
#0 lore raw

v7 cover psp: Add support for dev-assoc/disassoc

From: Wei Wang <[email protected]>

The main purpose of this feature is to associate virtual devices like veth or netkit with a real PSP device, so we could provide PSP functionality to the application running with virtual devices.

A typical deployment that works with this feature is as follows:
Host Namespace:
psp_dev_local ←──physically linked──→ psp_dev_peer
(PSP device)
│
│ BPF on psp_dev_local ingress: bpf_redirect_peer() to nk_guest │
nk_host / veth_host
│
│ BPF on nk_host ingress: bpf_redirect_neigh() to psp_dev_local │
Guest Namespace (netns):
│
nk_guest / veth_guest
★ PSP application run here

Remote Namespace (_netns):
psp_dev_peer
★ PSP server application runs here

Note:
The general requirement for this feature to work:
For PSP to work correctly, the egress device at validate_xmit_skb() time must have psp_dev matching the association's psd. Any device stacking or traffic redirection that changes the egress device will cause either:
1. TX validation failure (SKB_DROP_REASON_PSP_OUTPUT) - fail-safe
2. RX policy failure after tx-assoc - packets without PSP extension are rejected by receiver expecting encrypted traffic

Here are a few examples that this feature would not work:
- Bonding with load balancing in round-robin, XOR, 802.3ad mode across multiple PSP devices, or mixed PSP and non-PSP devices
- Bonding with active-backup mode might work without PSP migration for failover case.
- ipvlan/macvlan in bridge mode would not work given packets are loopbacked locally without going through the PSP device.

Changes since v6:
- Remove the unused remote_addr, nk_guest_addr and import cmd in patch 5

Changes since v5:
- Remove module_exit() in patch 3

Changes since v4:
- Address compilation warning in patch 3
- Removed the call to psp_nl_has_listeners_any_ns() and check listeners when looping through netns in psp_nl_notify_dev() in patch 2. This makes sure we only send notification to netns that has listeners.

Changes since v3:
- Make nsid optional for dev-assoc/dev-disassoc operation, and use the ns user is in when it's not specified. Also added a test for this.
- Fix psp_nl_notify_dev() to compute the correct nsid relative to the listener's netns.
- Only register the new netdev event for psp dev cleanup upon the first successful dev-assoc operation.
- Change the following in selftest:
- Add CONFIG_NETKIT to driver/net's config
- Fall back to NetDrvEpEnv and run basic test cases if NetDrvContEnv does not load
- Use ksft_variants instead of psp_ip_ver_test_builder

Changes since v2:
- Change the newly added parameter to psp_device_get_and_lock() to admin in patch 1. Introduce 2 device check functions:
- psp_device_get_locked_admin() for dev-set and key-rotate
- psp_device_get_locked() for all other operations
Flip the logic for checking the dev_assoc_list accordingly in patch 2.
- Move psp_nl_notify_dev() before removing the dev from assoc_dev_list in psp_nl_dev_disassoc_doit() and correct the typo in commit msg in patch 2.
- Remove the threading and subprocess and some comment updates in patch 5.

Changes since v1:
- Update the first 4 patches to reflect the latest changes in
https://lore.kernel.org/netdev/[email protected]/
- Update patch 9 to add a param to NetDrvContEnv to control the loading of the tx forwarding bpf program

Wei Wang (5):
psp: add admin/non-admin version of psp_device_get_locked
psp: add new netlink cmd for dev-assoc and dev-disassoc
psp: add a new netdev event for dev unregister selftests/net: Add bpf skb forwarding program selftest/net: psp: Add test for dev-assoc/disassoc

 Documentation/netlink/specs/psp.yaml          |  71 ++-
 include/net/psp/types.h                       |  15 +
 include/uapi/linux/psp.h                      |  13 +
 net/psp/psp-nl-gen.c                          |  36 +-
 net/psp/psp-nl-gen.h                          |   4 +
 net/psp/psp.h                                 |   3 +-
 net/psp/psp_main.c                            |  96 +++-
 net/psp/psp_nl.c                              | 344 +++++++++++-
 tools/testing/selftests/drivers/net/config    |   1 +
 .../drivers/net/hw/nk_redirect.bpf.c          |  60 +++
 .../selftests/drivers/net/lib/py/env.py       |  54 +-
 tools/testing/selftests/drivers/net/psp.py    | 492 ++++++++++++++++--
 12 files changed, 1131 insertions(+), 58 deletions(-)
 create mode 100644 tools/testing/selftests/drivers/net/hw/nk_redirect.bpf.c
Signature · 1 line
2.52.0
author
#1 lore raw

PATCH v7 1/5 psp: add admin/non-admin version of psp_device_get_locked

From: Wei Wang <[email protected]>

Introduce 2 versions of psp_device_get_locked:
1. psp_device_get_locked_admin(): This version is used for operations that would change the status of the psd, and are currently used for dev-set nad key-rotation.
2. psp_device_get_locked(): This is the non-admin version, which are used for broader user issued operations including: dev-get, rx-assoc, tx-assoc, get-stats.

Following commit will be implementing both of the checks.

---
 Documentation/netlink/specs/psp.yaml |  4 ++--
 net/psp/psp-nl-gen.c                 |  4 ++--
 net/psp/psp-nl-gen.h                 |  2 ++
 net/psp/psp.h                        |  2 +-
 net/psp/psp_main.c                   |  7 ++++++-
 net/psp/psp_nl.c                     | 28 +++++++++++++++++++++-------
 6 files changed, 34 insertions(+), 13 deletions(-)
Documentation/netlink/specs/psp.yaml +2 -2
index f3a57782d2cf..fe2cdc966604 100644
@@ -195,7 +195,7 @@ operations:
             - psp-versions-ena
         reply:
           attributes: []
-        pre: psp-device-get-locked
+        pre: psp-device-get-locked-admin
         post: psp-device-unlock
     -
       name: dev-change-ntf
@@ -214,7 +214,7 @@ operations:
         reply:
           attributes:
             - id
-        pre: psp-device-get-locked
+        pre: psp-device-get-locked-admin
         post: psp-device-unlock
     -
       name: key-rotate-ntf
net/psp/psp-nl-gen.c +2 -2
index 22a48d0fa378..1f5e73e7ccc1 100644
@@ -71,7 +71,7 @@ static const struct genl_split_ops psp_nl_ops[] = {
 	},
 	{
 		.cmd		= PSP_CMD_DEV_SET,
-		.pre_doit	= psp_device_get_locked,
+		.pre_doit	= psp_device_get_locked_admin,
 		.doit		= psp_nl_dev_set_doit,
 		.post_doit	= psp_device_unlock,
 		.policy		= psp_dev_set_nl_policy,
@@ -80,7 +80,7 @@ static const struct genl_split_ops psp_nl_ops[] = {
 	},
 	{
 		.cmd		= PSP_CMD_KEY_ROTATE,
-		.pre_doit	= psp_device_get_locked,
+		.pre_doit	= psp_device_get_locked_admin,
 		.doit		= psp_nl_key_rotate_doit,
 		.post_doit	= psp_device_unlock,
 		.policy		= psp_key_rotate_nl_policy,
net/psp/psp-nl-gen.h +2 -0
index 599c5f1c82f2..977355455395 100644
@@ -17,6 +17,8 @@ extern const struct nla_policy psp_keys_nl_policy[PSP_A_KEYS_SPI + 1];
 
 int psp_device_get_locked(const struct genl_split_ops *ops,
 			  struct sk_buff *skb, struct genl_info *info);
+int psp_device_get_locked_admin(const struct genl_split_ops *ops,
+				struct sk_buff *skb, struct genl_info *info);
 int psp_assoc_device_get_locked(const struct genl_split_ops *ops,
 				struct sk_buff *skb, struct genl_info *info);
 void
net/psp/psp.h +1 -1
index 9f19137593a0..0f9c4e4e52cb 100644
@@ -14,7 +14,7 @@ extern struct xarray psp_devs;
 extern struct mutex psp_devs_lock;
 
 void psp_dev_free(struct psp_dev *psd);
-int psp_dev_check_access(struct psp_dev *psd, struct net *net);
+int psp_dev_check_access(struct psp_dev *psd, struct net *net, bool admin);
 
 void psp_nl_notify_dev(struct psp_dev *psd, u32 cmd);
 
net/psp/psp_main.c +6 -1
index 9508b6c38003..82de78a1d6bd 100644
@@ -27,10 +27,15 @@ struct mutex psp_devs_lock;
  * psp_dev_check_access() - check if user in a given net ns can access PSP dev
  * @psd:	PSP device structure user is trying to access
  * @net:	net namespace user is in
+ * @admin:	If true, only allow access from @psd's main device's netns,
+ *		for admin operations like config changes and key rotation.
+ *		If false, also allow access from network namespaces that have
+ *		an associated device with @psd, for read-only and association
+ *		management operations.
  *
  * Return: 0 if PSP device should be visible in @net, errno otherwise.
  */
-int psp_dev_check_access(struct psp_dev *psd, struct net *net)
+int psp_dev_check_access(struct psp_dev *psd, struct net *net, bool admin)
 {
 	if (dev_net(psd->main_netdev) == net)
 		return 0;
net/psp/psp_nl.c +21 -7
index 6afd7707ec12..b988f35412df 100644
@@ -41,7 +41,8 @@ static int psp_nl_reply_send(struct sk_buff *rsp, struct genl_info *info)
 /* Device stuff */
 
 static struct psp_dev *
-psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)
+psp_device_get_and_lock(struct net *net, struct nlattr *dev_id,
+			bool admin)
 {
 	struct psp_dev *psd;
 	int err;
@@ -56,7 +57,7 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)
 	mutex_lock(&psd->lock);
 	mutex_unlock(&psp_devs_lock);
 
-	err = psp_dev_check_access(psd, net);
+	err = psp_dev_check_access(psd, net, admin);
 	if (err) {
 		mutex_unlock(&psd->lock);
 		return ERR_PTR(err);
@@ -65,6 +66,18 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)
 	return psd;
 }
 
+int psp_device_get_locked_admin(const struct genl_split_ops *ops,
+				struct sk_buff *skb, struct genl_info *info)
+{
+	if (GENL_REQ_ATTR_CHECK(info, PSP_A_DEV_ID))
+		return -EINVAL;
+
+	info->user_ptr[0] = psp_device_get_and_lock(genl_info_net(info),
+						    info->attrs[PSP_A_DEV_ID],
+						    true);
+	return PTR_ERR_OR_ZERO(info->user_ptr[0]);
+}
+
 int psp_device_get_locked(const struct genl_split_ops *ops,
 			  struct sk_buff *skb, struct genl_info *info)
 {
@@ -72,7 +85,8 @@ int psp_device_get_locked(const struct genl_split_ops *ops,
 		return -EINVAL;
 
 	info->user_ptr[0] = psp_device_get_and_lock(genl_info_net(info),
-						    info->attrs[PSP_A_DEV_ID]);
+						    info->attrs[PSP_A_DEV_ID],
+						    false);
 	return PTR_ERR_OR_ZERO(info->user_ptr[0]);
 }
 
@@ -160,7 +174,7 @@ static int
 psp_nl_dev_get_dumpit_one(struct sk_buff *rsp, struct netlink_callback *cb,
 			  struct psp_dev *psd)
 {
-	if (psp_dev_check_access(psd, sock_net(rsp->sk)))
+	if (psp_dev_check_access(psd, sock_net(rsp->sk), false))
 		return 0;
 
 	return psp_nl_dev_fill(psd, rsp, genl_info_dump(cb));
@@ -305,7 +319,7 @@ int psp_assoc_device_get_locked(const struct genl_split_ops *ops,
 
 	psd = psp_dev_get_for_sock(socket->sk);
 	if (psd) {
-		err = psp_dev_check_access(psd, genl_info_net(info));
+		err = psp_dev_check_access(psd, genl_info_net(info), false);
 		if (err) {
 			psp_dev_put(psd);
 			psd = NULL;
@@ -330,7 +344,7 @@ int psp_assoc_device_get_locked(const struct genl_split_ops *ops,
 
 		psp_dev_put(psd);
 	} else {
-		psd = psp_device_get_and_lock(genl_info_net(info), id);
+		psd = psp_device_get_and_lock(genl_info_net(info), id, false);
 		if (IS_ERR(psd)) {
 			err = PTR_ERR(psd);
 			goto err_sock_put;
@@ -573,7 +587,7 @@ static int
 psp_nl_stats_get_dumpit_one(struct sk_buff *rsp, struct netlink_callback *cb,
 			    struct psp_dev *psd)
 {
-	if (psp_dev_check_access(psd, sock_net(rsp->sk)))
+	if (psp_dev_check_access(psd, sock_net(rsp->sk), false))
 		return 0;
 
 	return psp_nl_stats_fill(psd, rsp, genl_info_dump(cb));
Signature · 1 line
2.52.0
author
#4 lore raw

PATCH v7 2/5 psp: add new netlink cmd for dev-assoc and dev-disassoc

In reply to Wei Wang

From: Wei Wang <[email protected]>

The main purpose of this cmd is to be able to associate a non-psp-capable device (e.g. veth or netkit) with a psp device. One use case is if we create a pair of veth/netkit, and assign 1 end inside a netns, while leaving the other end within the default netns, with a real PSP device, e.g. netdevsim or a physical PSP-capable NIC. With this command, we could associate the veth/netkit inside the netns with PSP device, so the virtual device could act as PSP-capable device to initiate PSP connections, and performs PSP encryption/decryption on the real PSP device.

---
 Documentation/netlink/specs/psp.yaml |  67 +++++-
 include/net/psp/types.h              |  15 ++
 include/uapi/linux/psp.h             |  13 ++
 net/psp/psp-nl-gen.c                 |  32 +++
 net/psp/psp-nl-gen.h                 |   2 +
 net/psp/psp_main.c                   |  21 +-
 net/psp/psp_nl.c                     | 309 ++++++++++++++++++++++++++-
 7 files changed, 447 insertions(+), 12 deletions(-)
Documentation/netlink/specs/psp.yaml +66 -1
index fe2cdc966604..336ef19155ff 100644
@@ -13,6 +13,17 @@ definitions:
               hdr0-aes-gmac-128, hdr0-aes-gmac-256]
 
 attribute-sets:
+  -
+    name: assoc-dev-info
+    attributes:
+      -
+        name: ifindex
+        doc: ifindex of an associated network device.
+        type: u32
+      -
+        name: nsid
+        doc: Network namespace ID of the associated device.
+        type: s32
   -
     name: dev
     attributes:
@@ -24,7 +35,9 @@ attribute-sets:
           min: 1
       -
         name: ifindex
-        doc: ifindex of the main netdevice linked to the PSP device.
+        doc: |
+          ifindex of the main netdevice linked to the PSP device,
+          or the ifindex to associate with the PSP device.
         type: u32
       -
         name: psp-versions-cap
@@ -38,6 +51,28 @@ attribute-sets:
         type: u32
         enum: version
         enum-as-flags: true
+      -
+        name: assoc-list
+        doc: List of associated virtual devices.
+        type: nest
+        nested-attributes: assoc-dev-info
+        multi-attr: true
+      -
+        name: nsid
+        doc: |
+          Network namespace ID for the device to associate/disassociate.
+          Optional for dev-assoc and dev-disassoc; if not present, the
+          device is looked up in the caller's network namespace.
+        type: s32
+      -
+        name: by-association
+        doc: |
+          Flag indicating the PSP device is an associated device from a
+          different network namespace.
+          Present when in associated namespace, absent when in primary/host
+          namespace.
+        type: flag
+
   -
     name: assoc
     attributes:
@@ -170,6 +205,8 @@ operations:
             - ifindex
             - psp-versions-cap
             - psp-versions-ena
+            - assoc-list
+            - by-association
         pre: psp-device-get-locked
         post: psp-device-unlock
       dump:
@@ -271,6 +308,34 @@ operations:
         post: psp-device-unlock
       dump:
         reply: *stats-all
+    -
+      name: dev-assoc
+      doc: Associate a network device with a PSP device.
+      attribute-set: dev
+      do:
+        request:
+          attributes:
+            - id
+            - ifindex
+            - nsid
+        reply:
+          attributes: []
+        pre: psp-device-get-locked
+        post: psp-device-unlock
+    -
+      name: dev-disassoc
+      doc: Disassociate a network device from a PSP device.
+      attribute-set: dev
+      do:
+        request:
+          attributes:
+            - id
+            - ifindex
+            - nsid
+        reply:
+          attributes: []
+        pre: psp-device-get-locked
+        post: psp-device-unlock
 
 mcast-groups:
   list:
include/net/psp/types.h +15 -0
index 25a9096d4e7d..4bd432ed107a 100644
@@ -5,6 +5,7 @@
 
 #include <linux/mutex.h>
 #include <linux/refcount.h>
+#include <net/net_trackers.h>
 
 struct netlink_ext_ack;
 
@@ -43,9 +44,22 @@ struct psp_dev_config {
 	u32 versions;
 };
 
+/**
+ * struct psp_assoc_dev - wrapper for associated net_device
+ * @dev_list: list node for psp_dev::assoc_dev_list
+ * @assoc_dev: the associated net_device
+ * @dev_tracker: tracker for the net_device reference
+ */
+struct psp_assoc_dev {
+	struct list_head dev_list;
+	struct net_device *assoc_dev;
+	netdevice_tracker dev_tracker;
+};
+
 /**
  * struct psp_dev - PSP device struct
  * @main_netdev: original netdevice of this PSP device
+ * @assoc_dev_list: list of psp_assoc_dev entries associated with this PSP device
  * @ops:	driver callbacks
  * @caps:	device capabilities
  * @drv_priv:	driver priv pointer
@@ -67,6 +81,7 @@ struct psp_dev_config {
  */
 struct psp_dev {
 	struct net_device *main_netdev;
+	struct list_head assoc_dev_list;
 
 	struct psp_dev_ops *ops;
 	struct psp_dev_caps *caps;
include/uapi/linux/psp.h +13 -0
index a3a336488dc3..1c8899cd4da5 100644
@@ -17,11 +17,22 @@ enum psp_version {
 	PSP_VERSION_HDR0_AES_GMAC_256,
 };
 
+enum {
+	PSP_A_ASSOC_DEV_INFO_IFINDEX = 1,
+	PSP_A_ASSOC_DEV_INFO_NSID,
+
+	__PSP_A_ASSOC_DEV_INFO_MAX,
+	PSP_A_ASSOC_DEV_INFO_MAX = (__PSP_A_ASSOC_DEV_INFO_MAX - 1)
+};
+
 enum {
 	PSP_A_DEV_ID = 1,
 	PSP_A_DEV_IFINDEX,
 	PSP_A_DEV_PSP_VERSIONS_CAP,
 	PSP_A_DEV_PSP_VERSIONS_ENA,
+	PSP_A_DEV_ASSOC_LIST,
+	PSP_A_DEV_NSID,
+	PSP_A_DEV_BY_ASSOCIATION,
 
 	__PSP_A_DEV_MAX,
 	PSP_A_DEV_MAX = (__PSP_A_DEV_MAX - 1)
@@ -74,6 +85,8 @@ enum {
 	PSP_CMD_RX_ASSOC,
 	PSP_CMD_TX_ASSOC,
 	PSP_CMD_GET_STATS,
+	PSP_CMD_DEV_ASSOC,
+	PSP_CMD_DEV_DISASSOC,
 
 	__PSP_CMD_MAX,
 	PSP_CMD_MAX = (__PSP_CMD_MAX - 1)
net/psp/psp-nl-gen.c +32 -0
index 1f5e73e7ccc1..114299c64423 100644
@@ -53,6 +53,20 @@ static const struct nla_policy psp_get_stats_nl_policy[PSP_A_STATS_DEV_ID + 1] =
 	[PSP_A_STATS_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1),
 };
 
+/* PSP_CMD_DEV_ASSOC - do */
+static const struct nla_policy psp_dev_assoc_nl_policy[PSP_A_DEV_NSID + 1] = {
+	[PSP_A_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1),
+	[PSP_A_DEV_IFINDEX] = { .type = NLA_U32, },
+	[PSP_A_DEV_NSID] = { .type = NLA_S32, },
+};
+
+/* PSP_CMD_DEV_DISASSOC - do */
+static const struct nla_policy psp_dev_disassoc_nl_policy[PSP_A_DEV_NSID + 1] = {
+	[PSP_A_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1),
+	[PSP_A_DEV_IFINDEX] = { .type = NLA_U32, },
+	[PSP_A_DEV_NSID] = { .type = NLA_S32, },
+};
+
 /* Ops table for psp */
 static const struct genl_split_ops psp_nl_ops[] = {
 	{
@@ -119,6 +133,24 @@ static const struct genl_split_ops psp_nl_ops[] = {
 		.dumpit	= psp_nl_get_stats_dumpit,
 		.flags	= GENL_CMD_CAP_DUMP,
 	},
+	{
+		.cmd		= PSP_CMD_DEV_ASSOC,
+		.pre_doit	= psp_device_get_locked,
+		.doit		= psp_nl_dev_assoc_doit,
+		.post_doit	= psp_device_unlock,
+		.policy		= psp_dev_assoc_nl_policy,
+		.maxattr	= PSP_A_DEV_NSID,
+		.flags		= GENL_CMD_CAP_DO,
+	},
+	{
+		.cmd		= PSP_CMD_DEV_DISASSOC,
+		.pre_doit	= psp_device_get_locked,
+		.doit		= psp_nl_dev_disassoc_doit,
+		.post_doit	= psp_device_unlock,
+		.policy		= psp_dev_disassoc_nl_policy,
+		.maxattr	= PSP_A_DEV_NSID,
+		.flags		= GENL_CMD_CAP_DO,
+	},
 };
 
 static const struct genl_multicast_group psp_nl_mcgrps[] = {
net/psp/psp-nl-gen.h +2 -0
index 977355455395..4dd0f0f23053 100644
@@ -33,6 +33,8 @@ int psp_nl_rx_assoc_doit(struct sk_buff *skb, struct genl_info *info);
 int psp_nl_tx_assoc_doit(struct sk_buff *skb, struct genl_info *info);
 int psp_nl_get_stats_doit(struct sk_buff *skb, struct genl_info *info);
 int psp_nl_get_stats_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
+int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info);
+int psp_nl_dev_disassoc_doit(struct sk_buff *skb, struct genl_info *info);
 
 enum {
 	PSP_NLGRP_MGMT,
net/psp/psp_main.c +20 -1
index 82de78a1d6bd..178b848989f1 100644
@@ -37,8 +37,18 @@ struct mutex psp_devs_lock;
  */
 int psp_dev_check_access(struct psp_dev *psd, struct net *net, bool admin)
 {
+	struct psp_assoc_dev *entry;
+
 	if (dev_net(psd->main_netdev) == net)
 		return 0;
+
+	if (!admin) {
+		list_for_each_entry(entry, &psd->assoc_dev_list, dev_list) {
+			if (dev_net(entry->assoc_dev) == net)
+				return 0;
+		}
+	}
+
 	return -ENOENT;
 }
 
@@ -74,6 +84,7 @@ psp_dev_create(struct net_device *netdev,
 		return ERR_PTR(-ENOMEM);
 
 	psd->main_netdev = netdev;
+	INIT_LIST_HEAD(&psd->assoc_dev_list);
 	psd->ops = psd_ops;
 	psd->caps = psd_caps;
 	psd->drv_priv = priv_ptr;
@@ -121,6 +132,7 @@ void psp_dev_free(struct psp_dev *psd)
  */
 void psp_dev_unregister(struct psp_dev *psd)
 {
+	struct psp_assoc_dev *entry, *entry_tmp;
 	struct psp_assoc *pas, *next;
 
 	mutex_lock(&psp_devs_lock);
@@ -140,6 +152,14 @@ void psp_dev_unregister(struct psp_dev *psd)
 	list_for_each_entry_safe(pas, next, &psd->stale_assocs, assocs_list)
 		psp_dev_tx_key_del(psd, pas);
 
+	list_for_each_entry_safe(entry, entry_tmp, &psd->assoc_dev_list,
+				 dev_list) {
+		list_del(&entry->dev_list);
+		rcu_assign_pointer(entry->assoc_dev->psp_dev, NULL);
+		netdev_put(entry->assoc_dev, &entry->dev_tracker);
+		kfree(entry);
+	}
+
 	rcu_assign_pointer(psd->main_netdev->psp_dev, NULL);
 
 	psd->ops = NULL;
@@ -361,5 +381,4 @@ static int __init psp_init(void)
 
 	return genl_register_family(&psp_nl_family);
 }
-
 subsys_initcall(psp_init);
net/psp/psp_nl.c +299 -10

369 lines not on this page · raw message

Signature · 1 line
2.52.0
author
#7 lore raw

PATCH v7 3/5 psp: add a new netdev event for dev unregister

In reply to Wei Wang

From: Wei Wang <[email protected]>

Add a new netdev event for dev unregister and handle the removal of this dev from psp->assoc_dev_list, upon the first successful dev-assoc operation.

---
 net/psp/psp.h      |  1 +
 net/psp/psp_main.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++
 net/psp/psp_nl.c   |  7 +++++
 3 files changed, 78 insertions(+)
net/psp/psp.h +1 -0
index 0f9c4e4e52cb..fd7457dedd30 100644
@@ -15,6 +15,7 @@ extern struct mutex psp_devs_lock;
 
 void psp_dev_free(struct psp_dev *psd);
 int psp_dev_check_access(struct psp_dev *psd, struct net *net, bool admin);
+void psp_attach_netdev_notifier(void);
 
 void psp_nl_notify_dev(struct psp_dev *psd, u32 cmd);
 
net/psp/psp_main.c +70 -0

80 lines not on this page · raw message

net/psp/psp_nl.c +7 -0
index aa60a8277829..44e00add4211 100644
@@ -515,6 +515,13 @@ int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info)
 
 	psp_nl_notify_dev(psd, PSP_CMD_DEV_CHANGE_NTF);
 
+	/* Register netdev notifier for assoc cleanup on success.
+	 * Must drop psd->lock to ensure lock ordering: rtnl_lock -> psd->lock
+	 */
+	mutex_unlock(&psd->lock);
+	psp_attach_netdev_notifier();
+	mutex_lock(&psd->lock);
+
 	return psp_nl_reply_send(rsp, info);
 }
 
Signature · 1 line
2.52.0
author
#10 lore raw

PATCH v7 4/5 selftests/net: Add bpf skb forwarding program

In reply to Wei Wang

From: Wei Wang <[email protected]>

Add nk_redirect.bpf.c, a BPF program that forwards skbs matching some IPv6 prefix received on eth0 ifindex to a specified dev ifindex. bpf_redirect_neigh() is used to make sure neighbor lookup is performed and proper MAC addr is being used.

---
 .../drivers/net/hw/nk_redirect.bpf.c          | 60 +++++++++++++++++++
 1 file changed, 60 insertions(+)
 create mode 100644 tools/testing/selftests/drivers/net/hw/nk_redirect.bpf.c
tools/testing/selftests/drivers/net/hw/nk_redirect.bpf.c new file +60 -0
new file mode 100644
index 000000000000..7ac9ffd50f15
@@ -0,0 +1,60 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * BPF program for redirecting traffic using bpf_redirect_neigh().
+ * Unlike bpf_redirect() which preserves L2 headers, bpf_redirect_neigh()
+ * performs neighbor lookup and fills in the correct L2 addresses for the
+ * target interface. This is necessary when redirecting across different
+ * device types (e.g., from netdevsim to netkit).
+ */
+#include <linux/bpf.h>
+#include <linux/pkt_cls.h>
+#include <linux/if_ether.h>
+#include <linux/ipv6.h>
+#include <linux/in6.h>
+#include <bpf/bpf_endian.h>
+#include <bpf/bpf_helpers.h>
+
+#define TC_ACT_OK 0
+#define ETH_P_IPV6 0x86DD
+
+#define ctx_ptr(field)		((void *)(long)(field))
+
+#define v6_p64_equal(a, b)	(a.s6_addr32[0] == b.s6_addr32[0] && \
+				 a.s6_addr32[1] == b.s6_addr32[1])
+
+volatile __u32 redirect_ifindex;
+volatile __u8 ipv6_prefix[16];
+
+SEC("tc/ingress")
+int tc_redirect(struct __sk_buff *skb)
+{
+	void *data_end = ctx_ptr(skb->data_end);
+	void *data = ctx_ptr(skb->data);
+	struct in6_addr *match_prefix;
+	struct ipv6hdr *ip6h;
+	struct ethhdr *eth;
+
+	match_prefix = (struct in6_addr *)ipv6_prefix;
+
+	if (skb->protocol != bpf_htons(ETH_P_IPV6))
+		return TC_ACT_OK;
+
+	eth = data;
+	if ((void *)(eth + 1) > data_end)
+		return TC_ACT_OK;
+
+	ip6h = data + sizeof(struct ethhdr);
+	if ((void *)(ip6h + 1) > data_end)
+		return TC_ACT_OK;
+
+	if (!v6_p64_equal(ip6h->daddr, (*match_prefix)))
+		return TC_ACT_OK;
+
+	/*
+	 * Use bpf_redirect_neigh() to perform neighbor lookup and fill in
+	 * correct L2 addresses for the target interface.
+	 */
+	return bpf_redirect_neigh(redirect_ifindex, NULL, 0, 0);
+}
+
+char __license[] SEC("license") = "GPL";
Signature · 1 line
2.52.0
author
#11 lore raw

PATCH v7 5/5 selftest/net: psp: Add test for dev-assoc/disassoc

In reply to Wei Wang

From: Wei Wang <[email protected]>

Add a new param to NetDrvContEnv to add an additional bpf redirect program on nk_host to redirect traffic to the psp_dev_local. The topology looks like this:
Host NS: psp_dev_local <---> nk_host

                |                |
                |                | (netkit pair)
                |                |
  Remote NS: psp_dev_peer      Guest NS: nk_guest
             (responder)             (PSP tests)

Add following tests for dev-assoc/dev-disassoc functionality:
1. Test the output of ./tools/net/ynl/pyynl/cli.py --spec Documentation/netlink/specs/psp.yaml --dump dev-get in both default and the guest netns.
2. Test the case where we associate netkit with psp_dev_local, and send PSP traffic from nk_guest to psp_dev_peer in 2 different netns.
3. Test to make sure the key rotation notification is sent to the netns for associated dev as well
4. Test to make sure the dev change notification is sent to the netns for associated dev as well
5. Test for dev-assoc/dev-disassoc without nsid parameter.
6. Test the deletion of nk_guest in client netns, and proper cleanup in the assoc-list for psp dev.

---
 tools/testing/selftests/drivers/net/config    |   1 +
 .../selftests/drivers/net/lib/py/env.py       |  54 +-
 tools/testing/selftests/drivers/net/psp.py    | 492 ++++++++++++++++--
 3 files changed, 513 insertions(+), 34 deletions(-)
tools/testing/selftests/drivers/net/config +1 -0
index 77ccf83d87e0..cdde8234dc07 100644
@@ -7,4 +7,5 @@ CONFIG_NETCONSOLE=m
 CONFIG_NETCONSOLE_DYNAMIC=y
 CONFIG_NETCONSOLE_EXTENDED_LOG=y
 CONFIG_NETDEVSIM=m
+CONFIG_NETKIT=y
 CONFIG_XDP_SOCKETS=y
tools/testing/selftests/drivers/net/lib/py/env.py +53 -1

82 lines not on this page · raw message

tools/testing/selftests/drivers/net/psp.py +459 -33

541 lines not on this page · raw message

Signature · 1 line
2.52.0
reviewer
#2 lore raw

Re: [PATCH v7 net-next 1/5] psp: add admin/non-admin version of psp_device_get_locked

In reply to Wei Wang

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.

For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add admin/non-admin version of psp_device_get_locked

This commit introduces two versions of psp_device_get_locked to distinguish between admin operations (dev-set, key-rotation) and non-admin operations (dev-get, rx-assoc, tx-assoc, get-stats).

Quoted 4 lines
Documentation/netlink/specs/psp.yaml +0 -0
index f3a57782d2cf..fe2cdc966604 100644

[ ... ]

Quoted 4 lines
net/psp/psp-nl-gen.h +0 -0
index 599c5f1c82f2..977355455395 100644

[ ... ]

Quoted 4 lines
net/psp/psp.h +0 -0
index 9f19137593a0..0f9c4e4e52cb 100644

[ ... ]

Quoted 4 lines
net/psp/psp_main.c +0 -0
index 9508b6c38003..82de78a1d6bd 100644

[ ... ]

Quoted 43 lines
net/psp/psp_nl.c +15 -2
index 6afd7707ec12..b988f35412df 100644
@@ -41,7 +41,8 @@ static int psp_nl_reply_send(struct sk_buff *rsp, struct genl_info *info)
 /* Device stuff */
 
 static struct psp_dev *
-psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)
+psp_device_get_and_lock(struct net *net, struct nlattr *dev_id,
+			bool admin)
 {
 	struct psp_dev *psd;
 	int err;
@@ -56,7 +57,7 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)
 	mutex_lock(&psd->lock);
 	mutex_unlock(&psp_devs_lock);
 
-	err = psp_dev_check_access(psd, net);
+	err = psp_dev_check_access(psd, net, admin);
 	if (err) {
 		mutex_unlock(&psd->lock);
 		return ERR_PTR(err);
@@ -65,6 +66,18 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)
 	return psd;
 }
 
+int psp_device_get_locked_admin(const struct genl_split_ops *ops,
+				struct sk_buff *skb, struct genl_info *info)
+{
+	if (GENL_REQ_ATTR_CHECK(info, PSP_A_DEV_ID))
+		return -EINVAL;
+
+	info->user_ptr[0] = psp_device_get_and_lock(genl_info_net(info),
+						    info->attrs[PSP_A_DEV_ID],
+						    true);
+	return PTR_ERR_OR_ZERO(info->user_ptr[0]);
+}
+
 int psp_device_get_locked(const struct genl_split_ops *ops,
 			  struct sk_buff *skb, struct genl_info *info)
 {

net/psp/psp_nl.c int psp_device_get_locked(const struct genl_split_ops *ops, · line 91

Quoted 8 lines
net/psp/psp_nl.c +2 -1
@@ -72,7 +85,8 @@ int psp_device_get_locked(const struct genl_split_ops *ops,
 		return -EINVAL;
 
 	info->user_ptr[0] = psp_device_get_and_lock(genl_info_net(info),
-						    info->attrs[PSP_A_DEV_ID]);
+						    info->attrs[PSP_A_DEV_ID],
+						    false);
 	return PTR_ERR_OR_ZERO(info->user_ptr[0]);
 }

This isn't a bug, but psp_device_get_locked_admin() and psp_device_get_locked() are identical except for the true/false parameter passed to psp_device_get_and_lock(). Would it be cleaner to introduce a common helper like __psp_device_get_locked(bool admin) to avoid the duplication?

Reference: https://lore.kernel.org/netdev/[email protected]/
Daniel Zahka raised this refactoring suggestion in v2, noting the two functions are entirely identical modulo the admin parameter.

Quoted 9 lines
net/psp/psp_nl.c +1 -1
@@ -160,7 +174,7 @@ static int
 psp_nl_dev_get_dumpit_one(struct sk_buff *rsp, struct netlink_callback *cb,
 			  struct psp_dev *psd)
 {
-	if (psp_dev_check_access(psd, sock_net(rsp->sk)))
+	if (psp_dev_check_access(psd, sock_net(rsp->sk), false))
 		return 0;
 
 	return psp_nl_dev_fill(psd, rsp, genl_info_dump(cb));

[ ... ]

reviewer
#5 lore raw

Re: [PATCH v7 net-next 2/5] psp: add new netlink cmd for dev-assoc and dev-disassoc

In reply to Wei Wang

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.

For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add new netlink cmd for dev-assoc and dev-disassoc

This commit adds new netlink commands (PSP_CMD_DEV_ASSOC and PSP_CMD_DEV_DISASSOC) to associate non-PSP-capable devices (e.g. veth or netkit) with a PSP device, enabling them to act as PSP-capable devices by performing PSP encryption/decryption on the real PSP device.

Quoted 12 lines
net/psp/psp_main.c +2 -0
index 82de78a1d6bd..178b848989f1 100644
@@ -37,8 +37,18 @@ struct mutex psp_devs_lock;
  */
 int psp_dev_check_access(struct psp_dev *psd, struct net *net, bool admin)
 {
+	struct psp_assoc_dev *entry;
+
 	if (dev_net(psd->main_netdev) == net)
 		return 0;

net/psp/psp_main.c struct mutex psp_devs_lock; · line 53

Quoted 10 lines
net/psp/psp_main.c +8 -0
+
+	if (!admin) {
+		list_for_each_entry(entry, &psd->assoc_dev_list, dev_list) {
+			if (dev_net(entry->assoc_dev) == net)
+				return 0;
+		}
+	}
+
 	return -ENOENT;
 }

Does this list iteration have a race with psp_dev_unregister()?

The new list_for_each_entry iteration over psd->assoc_dev_list in psp_dev_check_access is called from psp_assoc_device_get_locked (net/psp/psp_nl.c:615) without holding psd->lock.

That caller obtains psd via psp_dev_get_for_sock which takes a refcount but does not acquire psd->lock before calling psp_dev_check_access(psd, ..., false). Since admin is false, the new code path is entered.

Concurrently, psp_dev_unregister holds psd->lock and performs list_del + kfree on the same assoc_dev_list entries:

net/psp/psp_main.c:psp_dev_unregister() {
	...
	list_for_each_entry_safe(entry, entry_tmp, &psd->assoc_dev_list,
				 dev_list) {
		list_del(&entry->dev_list);
		rcu_assign_pointer(entry->assoc_dev->psp_dev, NULL);
		netdev_put(entry->assoc_dev, &entry->dev_tracker);
		kfree(entry);
	}
	...
}

The list iterator in psp_dev_check_access can follow a next pointer into freed memory, causing a use-after-free.

Other callers (psp_device_get_and_lock, psp_nl_dev_get_dumpit, psp_nl_get_stats_dumpit) correctly hold psd->lock before calling this function, so the race is specific to the psp_assoc_device_get_locked path.

Quoted 4 lines
net/psp/psp_nl.c +0 -0
index b988f35412df..aa60a8277829 100644

[ ... ]

Quoted 58 lines
net/psp/psp_nl.c +54 -0
@@ -291,6 +446,140 @@ int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info)
 	return err;
 }
 
+int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info)
+{
+	struct psp_dev *psd = info->user_ptr[0];
+	struct psp_assoc_dev *psp_assoc_dev;
+	struct net_device *assoc_dev;
+	u32 assoc_ifindex;
+	struct sk_buff *rsp;
+	struct net *net;
+	int nsid;
+
+	if (GENL_REQ_ATTR_CHECK(info, PSP_A_DEV_IFINDEX))
+		return -EINVAL;
+
+	if (info->attrs[PSP_A_DEV_NSID]) {
+		nsid = nla_get_s32(info->attrs[PSP_A_DEV_NSID]);
+
+		net = get_net_ns_by_id(genl_info_net(info), nsid);
+		if (!net) {
+			NL_SET_BAD_ATTR(info->extack,
+					info->attrs[PSP_A_DEV_NSID]);
+			return -EINVAL;
+		}
+	} else {
+		net = get_net(genl_info_net(info));
+	}
+
+	psp_assoc_dev = kzalloc(sizeof(*psp_assoc_dev), GFP_KERNEL);
+	if (!psp_assoc_dev) {
+		put_net(net);
+		return -ENOMEM;
+	}
+
+	assoc_ifindex = nla_get_u32(info->attrs[PSP_A_DEV_IFINDEX]);
+	assoc_dev = netdev_get_by_index(net, assoc_ifindex,
+					&psp_assoc_dev->dev_tracker,
+					GFP_KERNEL);
+	if (!assoc_dev) {
+		put_net(net);
+		kfree(psp_assoc_dev);
+		NL_SET_BAD_ATTR(info->extack, info->attrs[PSP_A_DEV_IFINDEX]);
+		return -ENODEV;
+	}
+
+	/* Check if device is already associated with a PSP device */
+	if (rcu_access_pointer(assoc_dev->psp_dev)) {
+		NL_SET_ERR_MSG(info->extack,
+			       "Device already associated with a PSP device");
+		netdev_put(assoc_dev, &psp_assoc_dev->dev_tracker);
+		put_net(net);
+		kfree(psp_assoc_dev);
+		return -EBUSY;
+	}
+
+	psp_assoc_dev->assoc_dev = assoc_dev;

net/psp/psp_nl.c int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info) · line 512

Quoted 10 lines
net/psp/psp_nl.c +10 -0
+	rsp = psp_nl_reply_new(info);
+	if (!rsp) {
+		netdev_put(assoc_dev, &psp_assoc_dev->dev_tracker);
+		put_net(net);
+		kfree(psp_assoc_dev);
+		return -ENOMEM;
+	}
+
+	rcu_assign_pointer(assoc_dev->psp_dev, psd);
+	list_add_tail(&psp_assoc_dev->dev_list, &psd->assoc_dev_list);

Can two concurrent PSP_CMD_DEV_ASSOC operations for the same net_device but different PSP devices both succeed?

The function checks rcu_access_pointer(assoc_dev->psp_dev) to verify the target net_device is not already associated with a PSP device, then later writes rcu_assign_pointer(assoc_dev->psp_dev, psd) and adds the wrapper to psd->assoc_dev_list.

These operations are only serialized by psd->lock, which is a per-PSP-device mutex acquired by the pre_doit callback psp_device_get_locked(). With parallel_ops=true in the psp_nl_family (net/psp/psp-nl-gen.c:165), two concurrent PSP_CMD_DEV_ASSOC operations targeting different PSP devices (each holding their own distinct psd->lock) but specifying the same target ifindex can both pass the rcu_access_pointer() NULL check.

Both then call rcu_assign_pointer(assoc_dev->psp_dev, psd) and list_add_tail(), resulting in the same net_device being added to two different PSP devices' assoc_dev_list. The second rcu_assign_pointer silently overwrites the first.

This leaves the first PSP device's assoc_dev_list containing an entry whose net_device->psp_dev points to the wrong PSP device:

Thread A (PSP device psd_A)        Thread B (PSP device psd_B)
================================   ================================
mutex_lock(&psd_A->lock)           mutex_lock(&psd_B->lock)
rcu_access_pointer(dev->psp_dev)
  -> NULL, proceed                 rcu_access_pointer(dev->psp_dev)
                                     -> NULL, proceed
rcu_assign_pointer(dev->psp_dev,   rcu_assign_pointer(dev->psp_dev,
                   psd_A)                             psd_B)
list_add_tail(entry_A,             list_add_tail(entry_B,
              &psd_A->list)                      &psd_B->list)

A subsequent disassociation from the first PSP device would find its stale psp_assoc_dev entry, call rcu_assign_pointer(found->assoc_dev->psp_dev, NULL), and clear the pointer that now belongs to the second PSP device, corrupting cross-device state.

The data path check in psp_validate_xmit (rcu_access_pointer(dev->psp_dev) == pas->psd) can then fail incorrectly.

reviewer
#8 lore raw

Re: [PATCH v7 net-next 3/5] psp: add a new netdev event for dev unregister

In reply to Wei Wang

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.

For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add a new netdev event for dev unregister

This commit adds a netdev notifier to handle device unregistration events and clean up PSP device associations when the associated network device is removed.

Quoted 16 lines
net/psp/psp.h +1 -0
index 0f9c4e4e52cb..fd7457dedd30 100644
@@ -15,6 +15,7 @@ extern struct mutex psp_devs_lock;
 
 void psp_dev_free(struct psp_dev *psd);
 int psp_dev_check_access(struct psp_dev *psd, struct net *net, bool admin);
+void psp_attach_netdev_notifier(void);
 
 void psp_nl_notify_dev(struct psp_dev *psd, u32 cmd);
 
net/psp/psp_main.c +0 -0
index 178b848989f1..db4593e76fa7 100644

[ ... ]

Quoted 10 lines
net/psp/psp_main.c +10 -0
+/**
+ * psp_attach_netdev_notifier() - register netdev notifier on first use
+ *
+ * Register the netdevice notifier when the first device association
+ * is created. In many installations no associations will be created and
+ * the notifier won't be needed.
+ *
+ * Must be called without psd->lock held, due to lock ordering:
+ * rtnl_lock -> psd->lock (the notifier callback runs under rtnl_lock
+ * and takes psd->lock).

net/psp/psp_main.c int psp_dev_rcv(struct sk_buff *skb, u16 dev_id, u8 generation, bool strip_icv) · line 442

Quoted 10 lines
net/psp/psp_main.c +10 -0
+ */
+void psp_attach_netdev_notifier(void)
+{
+	if (READ_ONCE(psp_notifier_registered))
+		return;
+
+	mutex_lock(&psp_devs_lock);
+	if (!psp_notifier_registered) {
+		register_netdevice_notifier(&psp_netdev_notifier);
+		WRITE_ONCE(psp_notifier_registered, true);
                                                             ^^^^

What happens if register_netdevice_notifier() fails? The function ignores the return value and sets psp_notifier_registered to true anyway.

register_netdevice_notifier() can fail with errors like -ENOMEM or errors from replaying existing device registrations. If registration fails, psp_notifier_registered will be permanently set to true, preventing any future registration attempts.

This means the notifier will never be registered, so NETDEV_UNREGISTER events will be silently missed and PSP device associations will never be cleaned up on device removal.

Quoted 7 lines
net/psp/psp_main.c +4 -0
+	}
+	mutex_unlock(&psp_devs_lock);
+}
+
 static int __init psp_init(void)
 {
 	mutex_init(&psp_devs_lock);

[ ... ]

Quoted 5 lines
net/psp/psp_nl.c +0 -0
index aa60a8277829..44e00add4211 100644

net/psp/psp_nl.c int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info) · line 524

Quoted 10 lines
net/psp/psp_nl.c +7 -0
@@ -515,6 +515,13 @@ int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info)
 
 	psp_nl_notify_dev(psd, PSP_CMD_DEV_CHANGE_NTF);
 
+	/* Register netdev notifier for assoc cleanup on success.
+	 * Must drop psd->lock to ensure lock ordering: rtnl_lock -> psd->lock
+	 */
+	mutex_unlock(&psd->lock);
+	psp_attach_netdev_notifier();
+	mutex_lock(&psd->lock);
+

Can dropping psd->lock here create a use-after-free window?

The pre_doit callback psp_device_get_and_lock() obtains psd via xa_load() and holds psd->lock, but does NOT take a refcount on psd.

When psd->lock is dropped here, psp_dev_unregister() can acquire both psp_devs_lock and psd->lock, complete teardown, and call psp_dev_put() which drops the refcount to 0:

net/psp/psp_main.c:psp_dev_unregister() {
    ...
    mutex_lock(&psp_devs_lock);
    ...
    mutex_lock(&psd->lock);
    ...
    xa_erase(&psp_devs, psd->id);
    ...
    mutex_unlock(&psd->lock);
    mutex_unlock(&psp_devs_lock);

    psp_dev_put(psd);  // can trigger psp_dev_free()
}

This would trigger psp_dev_free() which calls mutex_destroy(&psd->lock) and kfree_rcu(psd, rcu).

When this function then calls mutex_lock(&psd->lock), it operates on a destroyed mutex and logically freed memory. The post_doit callback psp_device_unlock() would then call mutex_unlock(&psd->lock) on the same destroyed mutex.

Should a refcount be taken on psd before dropping the lock, or should the notifier registration be moved outside the doit handler?

Quoted 2 lines
net/psp/psp_nl.c +0 -0
 	return psp_nl_reply_send(rsp, info);
 }
author
#3 lore raw

Re: [PATCH v7 net-next 1/5] psp: add admin/non-admin version of psp_device_get_locked

In reply to Simon Horman
Quoted 98 lines · Simon Horman

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.

For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add admin/non-admin version of psp_device_get_locked

This commit introduces two versions of psp_device_get_locked to distinguish between admin operations (dev-set, key-rotation) and non-admin operations (dev-get, rx-assoc, tx-assoc, get-stats).

Quoted 4 lines
Documentation/netlink/specs/psp.yaml +0 -0
index f3a57782d2cf..fe2cdc966604 100644

[ ... ]

Quoted 4 lines
net/psp/psp-nl-gen.h +0 -0
index 599c5f1c82f2..977355455395 100644

[ ... ]

Quoted 4 lines
net/psp/psp.h +0 -0
index 9f19137593a0..0f9c4e4e52cb 100644

[ ... ]

Quoted 4 lines
net/psp/psp_main.c +0 -0
index 9508b6c38003..82de78a1d6bd 100644

[ ... ]

Quoted 43 lines
net/psp/psp_nl.c +15 -2
index 6afd7707ec12..b988f35412df 100644
@@ -41,7 +41,8 @@ static int psp_nl_reply_send(struct sk_buff *rsp, struct genl_info *info)
 /* Device stuff */
 
 static struct psp_dev *
-psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)
+psp_device_get_and_lock(struct net *net, struct nlattr *dev_id,
+                     bool admin)
 {
      struct psp_dev *psd;
      int err;
@@ -56,7 +57,7 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)
      mutex_lock(&psd->lock);
      mutex_unlock(&psp_devs_lock);
 
-     err = psp_dev_check_access(psd, net);
+     err = psp_dev_check_access(psd, net, admin);
      if (err) {
              mutex_unlock(&psd->lock);
              return ERR_PTR(err);
@@ -65,6 +66,18 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)
      return psd;
 }
 
+int psp_device_get_locked_admin(const struct genl_split_ops *ops,
+                             struct sk_buff *skb, struct genl_info *info)
+{
+     if (GENL_REQ_ATTR_CHECK(info, PSP_A_DEV_ID))
+             return -EINVAL;
+
+     info->user_ptr[0] = psp_device_get_and_lock(genl_info_net(info),
+                                                 info->attrs[PSP_A_DEV_ID],
+                                                 true);
+     return PTR_ERR_OR_ZERO(info->user_ptr[0]);
+}
+
 int psp_device_get_locked(const struct genl_split_ops *ops,
                        struct sk_buff *skb, struct genl_info *info)
 {

net/psp/psp_nl.c int psp_device_get_locked(const struct genl_split_ops *ops, · line 91

Quoted 8 lines
net/psp/psp_nl.c +2 -1
@@ -72,7 +85,8 @@ int psp_device_get_locked(const struct genl_split_ops *ops,
              return -EINVAL;
 
      info->user_ptr[0] = psp_device_get_and_lock(genl_info_net(info),
-                                                 info->attrs[PSP_A_DEV_ID]);
+                                                 info->attrs[PSP_A_DEV_ID],
+                                                 false);
      return PTR_ERR_OR_ZERO(info->user_ptr[0]);
 }

This isn't a bug, but psp_device_get_locked_admin() and psp_device_get_locked() are identical except for the true/false parameter passed to psp_device_get_and_lock(). Would it be cleaner to introduce a common helper like __psp_device_get_locked(bool admin) to avoid the duplication?

Quoted 3 lines

Reference: https://lore.kernel.org/netdev/[email protected]/
Daniel Zahka raised this refactoring suggestion in v2, noting the two functions are entirely identical modulo the admin parameter.

Ack. Addressing in the next version.

Quoted 12 lines
Quoted 10 lines
net/psp/psp_nl.c +1 -1
@@ -160,7 +174,7 @@ static int
 psp_nl_dev_get_dumpit_one(struct sk_buff *rsp, struct netlink_callback *cb,
                        struct psp_dev *psd)
 {
-     if (psp_dev_check_access(psd, sock_net(rsp->sk)))
+     if (psp_dev_check_access(psd, sock_net(rsp->sk), false))
              return 0;
 
      return psp_nl_dev_fill(psd, rsp, genl_info_dump(cb));

[ ... ]

author
#6 lore raw

Re: [PATCH v7 net-next 2/5] psp: add new netlink cmd for dev-assoc and dev-disassoc

In reply to Simon Horman
Quoted 61 lines · Simon Horman

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.

For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add new netlink cmd for dev-assoc and dev-disassoc

This commit adds new netlink commands (PSP_CMD_DEV_ASSOC and PSP_CMD_DEV_DISASSOC) to associate non-PSP-capable devices (e.g. veth or netkit) with a PSP device, enabling them to act as PSP-capable devices by performing PSP encryption/decryption on the real PSP device.

Quoted 12 lines
net/psp/psp_main.c +2 -0
index 82de78a1d6bd..178b848989f1 100644
@@ -37,8 +37,18 @@ struct mutex psp_devs_lock;
  */
 int psp_dev_check_access(struct psp_dev *psd, struct net *net, bool admin)
 {
+     struct psp_assoc_dev *entry;
+
      if (dev_net(psd->main_netdev) == net)
              return 0;

net/psp/psp_main.c struct mutex psp_devs_lock; · line 53

Quoted 10 lines
net/psp/psp_main.c +8 -0
+
+     if (!admin) {
+             list_for_each_entry(entry, &psd->assoc_dev_list, dev_list) {
+                     if (dev_net(entry->assoc_dev) == net)
+                             return 0;
+             }
+     }
+
      return -ENOENT;
 }

Does this list iteration have a race with psp_dev_unregister()?

The new list_for_each_entry iteration over psd->assoc_dev_list in psp_dev_check_access is called from psp_assoc_device_get_locked (net/psp/psp_nl.c:615) without holding psd->lock.

That caller obtains psd via psp_dev_get_for_sock which takes a refcount but does not acquire psd->lock before calling psp_dev_check_access(psd, ..., false). Since admin is false, the new code path is entered.

Concurrently, psp_dev_unregister holds psd->lock and performs list_del + kfree on the same assoc_dev_list entries:

net/psp/psp_main.c:psp_dev_unregister() {
        ...
        list_for_each_entry_safe(entry, entry_tmp, &psd->assoc_dev_list,
                                 dev_list) {
                list_del(&entry->dev_list);
                rcu_assign_pointer(entry->assoc_dev->psp_dev, NULL);
                netdev_put(entry->assoc_dev, &entry->dev_tracker);
                kfree(entry);
        }
        ...
}
Quoted 7 lines

The list iterator in psp_dev_check_access can follow a next pointer into freed memory, causing a use-after-free.

Other callers (psp_device_get_and_lock, psp_nl_dev_get_dumpit, psp_nl_get_stats_dumpit) correctly hold psd->lock before calling this function, so the race is specific to the psp_assoc_device_get_locked path.

Grabbing psd->lock before psp_dev_check_access () in the next versions.

Quoted 110 lines
Quoted 4 lines
net/psp/psp_nl.c +0 -0
index b988f35412df..aa60a8277829 100644

[ ... ]

Quoted 58 lines
net/psp/psp_nl.c +54 -0
@@ -291,6 +446,140 @@ int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info)
      return err;
 }
 
+int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info)
+{
+     struct psp_dev *psd = info->user_ptr[0];
+     struct psp_assoc_dev *psp_assoc_dev;
+     struct net_device *assoc_dev;
+     u32 assoc_ifindex;
+     struct sk_buff *rsp;
+     struct net *net;
+     int nsid;
+
+     if (GENL_REQ_ATTR_CHECK(info, PSP_A_DEV_IFINDEX))
+             return -EINVAL;
+
+     if (info->attrs[PSP_A_DEV_NSID]) {
+             nsid = nla_get_s32(info->attrs[PSP_A_DEV_NSID]);
+
+             net = get_net_ns_by_id(genl_info_net(info), nsid);
+             if (!net) {
+                     NL_SET_BAD_ATTR(info->extack,
+                                     info->attrs[PSP_A_DEV_NSID]);
+                     return -EINVAL;
+             }
+     } else {
+             net = get_net(genl_info_net(info));
+     }
+
+     psp_assoc_dev = kzalloc(sizeof(*psp_assoc_dev), GFP_KERNEL);
+     if (!psp_assoc_dev) {
+             put_net(net);
+             return -ENOMEM;
+     }
+
+     assoc_ifindex = nla_get_u32(info->attrs[PSP_A_DEV_IFINDEX]);
+     assoc_dev = netdev_get_by_index(net, assoc_ifindex,
+                                     &psp_assoc_dev->dev_tracker,
+                                     GFP_KERNEL);
+     if (!assoc_dev) {
+             put_net(net);
+             kfree(psp_assoc_dev);
+             NL_SET_BAD_ATTR(info->extack, info->attrs[PSP_A_DEV_IFINDEX]);
+             return -ENODEV;
+     }
+
+     /* Check if device is already associated with a PSP device */
+     if (rcu_access_pointer(assoc_dev->psp_dev)) {
+             NL_SET_ERR_MSG(info->extack,
+                            "Device already associated with a PSP device");
+             netdev_put(assoc_dev, &psp_assoc_dev->dev_tracker);
+             put_net(net);
+             kfree(psp_assoc_dev);
+             return -EBUSY;
+     }
+
+     psp_assoc_dev->assoc_dev = assoc_dev;

net/psp/psp_nl.c int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info) · line 512

Quoted 10 lines
net/psp/psp_nl.c +10 -0
+     rsp = psp_nl_reply_new(info);
+     if (!rsp) {
+             netdev_put(assoc_dev, &psp_assoc_dev->dev_tracker);
+             put_net(net);
+             kfree(psp_assoc_dev);
+             return -ENOMEM;
+     }
+
+     rcu_assign_pointer(assoc_dev->psp_dev, psd);
+     list_add_tail(&psp_assoc_dev->dev_list, &psd->assoc_dev_list);

Can two concurrent PSP_CMD_DEV_ASSOC operations for the same net_device but different PSP devices both succeed?

The function checks rcu_access_pointer(assoc_dev->psp_dev) to verify the target net_device is not already associated with a PSP device, then later writes rcu_assign_pointer(assoc_dev->psp_dev, psd) and adds the wrapper to psd->assoc_dev_list.

These operations are only serialized by psd->lock, which is a per-PSP-device mutex acquired by the pre_doit callback psp_device_get_locked(). With parallel_ops=true in the psp_nl_family (net/psp/psp-nl-gen.c:165), two concurrent PSP_CMD_DEV_ASSOC operations targeting different PSP devices (each holding their own distinct psd->lock) but specifying the same target ifindex can both pass the rcu_access_pointer() NULL check.

Both then call rcu_assign_pointer(assoc_dev->psp_dev, psd) and list_add_tail(), resulting in the same net_device being added to two different PSP devices' assoc_dev_list. The second rcu_assign_pointer silently overwrites the first.

This leaves the first PSP device's assoc_dev_list containing an entry whose net_device->psp_dev points to the wrong PSP device:

Thread A (PSP device psd_A)        Thread B (PSP device psd_B)
================================   ================================
mutex_lock(&psd_A->lock)           mutex_lock(&psd_B->lock)
rcu_access_pointer(dev->psp_dev)
  -> NULL, proceed                 rcu_access_pointer(dev->psp_dev)
                                     -> NULL, proceed
rcu_assign_pointer(dev->psp_dev,   rcu_assign_pointer(dev->psp_dev,
                   psd_A)                             psd_B)
list_add_tail(entry_A,             list_add_tail(entry_B,
              &psd_A->list)                      &psd_B->list)
Quoted 7 lines

A subsequent disassociation from the first PSP device would find its stale psp_assoc_dev entry, call rcu_assign_pointer(found->assoc_dev->psp_dev, NULL), and clear the pointer that now belongs to the second PSP device, corrupting cross-device state.

The data path check in psp_validate_xmit (rcu_access_pointer(dev->psp_dev) == pas->psd) can then fail incorrectly.

Changing to use cmpxchg() on assoc_dev->psp_dev in the next version.

author
#9 lore raw

Re: [PATCH v7 net-next 3/5] psp: add a new netdev event for dev unregister

In reply to Simon Horman
Quoted 56 lines · Simon Horman

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.

For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add a new netdev event for dev unregister

This commit adds a netdev notifier to handle device unregistration events and clean up PSP device associations when the associated network device is removed.

Quoted 16 lines
net/psp/psp.h +1 -0
index 0f9c4e4e52cb..fd7457dedd30 100644
@@ -15,6 +15,7 @@ extern struct mutex psp_devs_lock;
 
 void psp_dev_free(struct psp_dev *psd);
 int psp_dev_check_access(struct psp_dev *psd, struct net *net, bool admin);
+void psp_attach_netdev_notifier(void);
 
 void psp_nl_notify_dev(struct psp_dev *psd, u32 cmd);
 
net/psp/psp_main.c +0 -0
index 178b848989f1..db4593e76fa7 100644

[ ... ]

Quoted 10 lines
net/psp/psp_main.c +10 -0
+/**
+ * psp_attach_netdev_notifier() - register netdev notifier on first use
+ *
+ * Register the netdevice notifier when the first device association
+ * is created. In many installations no associations will be created and
+ * the notifier won't be needed.
+ *
+ * Must be called without psd->lock held, due to lock ordering:
+ * rtnl_lock -> psd->lock (the notifier callback runs under rtnl_lock
+ * and takes psd->lock).

net/psp/psp_main.c int psp_dev_rcv(struct sk_buff *skb, u16 dev_id, u8 generation, bool strip_icv) · line 442

Quoted 10 lines
net/psp/psp_main.c +10 -0
+ */
+void psp_attach_netdev_notifier(void)
+{
+     if (READ_ONCE(psp_notifier_registered))
+             return;
+
+     mutex_lock(&psp_devs_lock);
+     if (!psp_notifier_registered) {
+             register_netdevice_notifier(&psp_netdev_notifier);
+             WRITE_ONCE(psp_notifier_registered, true);
                                                             ^^^^

What happens if register_netdevice_notifier() fails? The function ignores the return value and sets psp_notifier_registered to true anyway.

Quoted 8 lines

register_netdevice_notifier() can fail with errors like -ENOMEM or errors from replaying existing device registrations. If registration fails, psp_notifier_registered will be permanently set to true, preventing any future registration attempts.

This means the notifier will never be registered, so NETDEV_UNREGISTER events will be silently missed and PSP device associations will never be cleaned up on device removal.

Adding the failing check in the next version.

Quoted 53 lines
Quoted 8 lines
net/psp/psp_main.c +4 -0
+     }
+     mutex_unlock(&psp_devs_lock);
+}
+
 static int __init psp_init(void)
 {
      mutex_init(&psp_devs_lock);

[ ... ]

Quoted 5 lines
net/psp/psp_nl.c +0 -0
index aa60a8277829..44e00add4211 100644

net/psp/psp_nl.c int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info) · line 524

Quoted 10 lines
net/psp/psp_nl.c +7 -0
@@ -515,6 +515,13 @@ int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info)
 
      psp_nl_notify_dev(psd, PSP_CMD_DEV_CHANGE_NTF);
 
+     /* Register netdev notifier for assoc cleanup on success.
+      * Must drop psd->lock to ensure lock ordering: rtnl_lock -> psd->lock
+      */
+     mutex_unlock(&psd->lock);
+     psp_attach_netdev_notifier();
+     mutex_lock(&psd->lock);
+

Can dropping psd->lock here create a use-after-free window?

The pre_doit callback psp_device_get_and_lock() obtains psd via xa_load() and holds psd->lock, but does NOT take a refcount on psd.

When psd->lock is dropped here, psp_dev_unregister() can acquire both psp_devs_lock and psd->lock, complete teardown, and call psp_dev_put() which drops the refcount to 0:

net/psp/psp_main.c:psp_dev_unregister() {
    ...
    mutex_lock(&psp_devs_lock);
    ...
    mutex_lock(&psd->lock);
    ...
    xa_erase(&psp_devs, psd->id);
    ...
    mutex_unlock(&psd->lock);
    mutex_unlock(&psp_devs_lock);

    psp_dev_put(psd);  // can trigger psp_dev_free()
}

This would trigger psp_dev_free() which calls mutex_destroy(&psd->lock) and kfree_rcu(psd, rcu).

Quoted 8 lines

When this function then calls mutex_lock(&psd->lock), it operates on a destroyed mutex and logically freed memory. The post_doit callback psp_device_unlock() would then call mutex_unlock(&psd->lock) on the same destroyed mutex.

Should a refcount be taken on psd before dropping the lock, or should the notifier registration be moved outside the doit handler?

Will move psp_attach_netdev_notifier() to pre_doit handler.

Quoted 2 lines
Quoted 2 lines
net/psp/psp_nl.c +0 -0
      return psp_nl_reply_send(rsp, info);
 }