psp: Add support for dev-assoc/disassoc
Subsystems (5)
v7 cover psp: Add support for dev-assoc/disassoc
From: Wei Wang <[email protected]>
The main purpose of this feature is to associate virtual devices like veth or netkit with a real PSP device, so we could provide PSP functionality to the application running with virtual devices.
A typical deployment that works with this feature is as follows:
Host Namespace:
psp_dev_local ←──physically linked──→ psp_dev_peer
(PSP device)
│
│ BPF on psp_dev_local ingress: bpf_redirect_peer() to nk_guest │
nk_host / veth_host
│
│ BPF on nk_host ingress: bpf_redirect_neigh() to psp_dev_local │
Guest Namespace (netns):
│
nk_guest / veth_guest
★ PSP application run here
Remote Namespace (_netns):
psp_dev_peer
★ PSP server application runs here
Note:
The general requirement for this feature to work:
For PSP to work correctly, the egress device at validate_xmit_skb() time must have psp_dev matching the association's psd. Any device stacking or traffic redirection that changes the egress device will cause either:
1. TX validation failure (SKB_DROP_REASON_PSP_OUTPUT) - fail-safe
2. RX policy failure after tx-assoc - packets without PSP extension are rejected by receiver expecting encrypted traffic
Here are a few examples that this feature would not work:
- Bonding with load balancing in round-robin, XOR, 802.3ad mode across multiple PSP devices, or mixed PSP and non-PSP devices
- Bonding with active-backup mode might work without PSP migration for failover case.
- ipvlan/macvlan in bridge mode would not work given packets are loopbacked locally without going through the PSP device.
Changes since v6:
- Remove the unused remote_addr, nk_guest_addr and import cmd in patch 5
Changes since v5:
- Remove module_exit() in patch 3
Changes since v4:
- Address compilation warning in patch 3
- Removed the call to psp_nl_has_listeners_any_ns() and check listeners when looping through netns in psp_nl_notify_dev() in patch 2. This makes sure we only send notification to netns that has listeners.
Changes since v3:
- Make nsid optional for dev-assoc/dev-disassoc operation, and use the ns user is in when it's not specified. Also added a test for this.
- Fix psp_nl_notify_dev() to compute the correct nsid relative to the listener's netns.
- Only register the new netdev event for psp dev cleanup upon the first successful dev-assoc operation.
- Change the following in selftest:
- Add CONFIG_NETKIT to driver/net's config
- Fall back to NetDrvEpEnv and run basic test cases if NetDrvContEnv does not load
- Use ksft_variants instead of psp_ip_ver_test_builder
Changes since v2:
- Change the newly added parameter to psp_device_get_and_lock() to admin in patch 1. Introduce 2 device check functions:
- psp_device_get_locked_admin() for dev-set and key-rotate
- psp_device_get_locked() for all other operations
Flip the logic for checking the dev_assoc_list accordingly in patch 2.
- Move psp_nl_notify_dev() before removing the dev from assoc_dev_list in psp_nl_dev_disassoc_doit() and correct the typo in commit msg in patch 2.
- Remove the threading and subprocess and some comment updates in patch 5.
Changes since v1:
- Update the first 4 patches to reflect the latest changes in
https://lore.kernel.org/netdev/[email protected]/
- Update patch 9 to add a param to NetDrvContEnv to control the loading of the tx forwarding bpf program
Wei Wang (5):
psp: add admin/non-admin version of psp_device_get_locked
psp: add new netlink cmd for dev-assoc and dev-disassoc
psp: add a new netdev event for dev unregister selftests/net: Add bpf skb forwarding program selftest/net: psp: Add test for dev-assoc/disassoc
Documentation/netlink/specs/psp.yaml | 71 ++- include/net/psp/types.h | 15 + include/uapi/linux/psp.h | 13 + net/psp/psp-nl-gen.c | 36 +- net/psp/psp-nl-gen.h | 4 + net/psp/psp.h | 3 +- net/psp/psp_main.c | 96 +++- net/psp/psp_nl.c | 344 +++++++++++- tools/testing/selftests/drivers/net/config | 1 + .../drivers/net/hw/nk_redirect.bpf.c | 60 +++ .../selftests/drivers/net/lib/py/env.py | 54 +- tools/testing/selftests/drivers/net/psp.py | 492 ++++++++++++++++-- 12 files changed, 1131 insertions(+), 58 deletions(-) create mode 100644 tools/testing/selftests/drivers/net/hw/nk_redirect.bpf.c
Signature · 1 line
2.52.0
PATCH v7 1/5 psp: add admin/non-admin version of psp_device_get_locked
From: Wei Wang <[email protected]>
Introduce 2 versions of psp_device_get_locked:
1. psp_device_get_locked_admin(): This version is used for operations that would change the status of the psd, and are currently used for dev-set nad key-rotation.
2. psp_device_get_locked(): This is the non-admin version, which are used for broader user issued operations including: dev-get, rx-assoc, tx-assoc, get-stats.
Following commit will be implementing both of the checks.
--- Documentation/netlink/specs/psp.yaml | 4 ++-- net/psp/psp-nl-gen.c | 4 ++-- net/psp/psp-nl-gen.h | 2 ++ net/psp/psp.h | 2 +- net/psp/psp_main.c | 7 ++++++- net/psp/psp_nl.c | 28 +++++++++++++++++++++------- 6 files changed, 34 insertions(+), 13 deletions(-)
Documentation/netlink/specs/psp.yaml +2 -2
@@ -195,7 +195,7 @@ operations: - psp-versions-ena reply: attributes: [] - pre: psp-device-get-locked + pre: psp-device-get-locked-admin post: psp-device-unlock - name: dev-change-ntf
@@ -214,7 +214,7 @@ operations: reply: attributes: - id - pre: psp-device-get-locked + pre: psp-device-get-locked-admin post: psp-device-unlock - name: key-rotate-ntf
net/psp/psp-nl-gen.c +2 -2
@@ -71,7 +71,7 @@ static const struct genl_split_ops psp_nl_ops[] = { }, { .cmd = PSP_CMD_DEV_SET, - .pre_doit = psp_device_get_locked, + .pre_doit = psp_device_get_locked_admin, .doit = psp_nl_dev_set_doit, .post_doit = psp_device_unlock, .policy = psp_dev_set_nl_policy,
@@ -80,7 +80,7 @@ static const struct genl_split_ops psp_nl_ops[] = { }, { .cmd = PSP_CMD_KEY_ROTATE, - .pre_doit = psp_device_get_locked, + .pre_doit = psp_device_get_locked_admin, .doit = psp_nl_key_rotate_doit, .post_doit = psp_device_unlock, .policy = psp_key_rotate_nl_policy,
net/psp/psp-nl-gen.h +2 -0
@@ -17,6 +17,8 @@ extern const struct nla_policy psp_keys_nl_policy[PSP_A_KEYS_SPI + 1];int psp_device_get_locked (const struct genl_split_ops *ops,struct sk_buff *skb,struct genl_info *info); +int psp_device_get_locked_admin (const struct genl_split_ops *ops, +struct sk_buff *skb,struct genl_info *info);int psp_assoc_device_get_locked (const struct genl_split_ops *ops,struct sk_buff *skb,struct genl_info *info);void
net/psp/psp.h +1 -1
@@ -14,7 +14,7 @@ extern struct xarray psp_devs;extern struct mutex psp_devs_lock;void psp_dev_free (struct psp_dev *psd); -int psp_dev_check_access (struct psp_dev *psd,struct net *net); +int psp_dev_check_access (struct psp_dev *psd,struct net *net,bool admin);void psp_nl_notify_dev (struct psp_dev *psd,u32 cmd);
net/psp/psp_main.c +6 -1
@@ -27,10 +27,15 @@ struct mutex psp_devs_lock;* psp_dev_check_access() - check if user in a given net ns can access PSP dev * @psd: PSP device structure user is trying to access * @net: net namespace user is in +* @admin: If true, only allow access from @psd's main device's netns, +* for admin operations like config changes and key rotation. +* If false, also allow access from network namespaces that have +* an associated device with @psd, for read-only and association +* management operations. * * Return: 0 if PSP device should be visible in @net, errno otherwise. */ -int psp_dev_check_access (struct psp_dev *psd,struct net *net) +int psp_dev_check_access (struct psp_dev *psd,struct net *net,bool admin) {if (dev_net (psd->main_netdev) == net)return 0 ;
net/psp/psp_nl.c +21 -7
@@ -41,7 +41,8 @@ static int psp_nl_reply_send(struct sk_buff *rsp, struct genl_info *info)/* Device stuff */ static struct psp_dev * -psp_device_get_and_lock (struct net *net,struct nlattr *dev_id) +psp_device_get_and_lock (struct net *net,struct nlattr *dev_id, +bool admin) {struct psp_dev *psd;int err;
@@ -56,7 +57,7 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)mutex_lock (&psd->lock);mutex_unlock (&psp_devs_lock); - err =psp_dev_check_access (psd, net); + err =psp_dev_check_access (psd, net, admin);if (err) {mutex_unlock (&psd->lock);return ERR_PTR (err);
@@ -65,6 +66,18 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)return psd; } +int psp_device_get_locked_admin (const struct genl_split_ops *ops, +struct sk_buff *skb,struct genl_info *info) +{ +if (GENL_REQ_ATTR_CHECK (info,PSP_A_DEV_ID )) +return -EINVAL ; + + info->user_ptr[0 ] =psp_device_get_and_lock (genl_info_net (info), + info->attrs[PSP_A_DEV_ID ], + true); +return PTR_ERR_OR_ZERO (info->user_ptr[0 ]); +} +int psp_device_get_locked (const struct genl_split_ops *ops,struct sk_buff *skb,struct genl_info *info) {
@@ -72,7 +85,8 @@ int psp_device_get_locked(const struct genl_split_ops *ops,return -EINVAL ; info->user_ptr[0 ] =psp_device_get_and_lock (genl_info_net (info), - info->attrs[PSP_A_DEV_ID ]); + info->attrs[PSP_A_DEV_ID ], + false);return PTR_ERR_OR_ZERO (info->user_ptr[0 ]); }
@@ -160,7 +174,7 @@ static int psp_nl_dev_get_dumpit_one(struct sk_buff *rsp, struct netlink_callback *cb, struct psp_dev *psd) { - if (psp_dev_check_access(psd, sock_net(rsp->sk))) + if (psp_dev_check_access(psd, sock_net(rsp->sk), false)) return 0; return psp_nl_dev_fill(psd, rsp, genl_info_dump(cb));
@@ -305,7 +319,7 @@ int psp_assoc_device_get_locked(const struct genl_split_ops *ops, psd =psp_dev_get_for_sock (socket->sk);if (psd) { - err =psp_dev_check_access (psd,genl_info_net (info)); + err =psp_dev_check_access (psd,genl_info_net (info), false);if (err) {psp_dev_put (psd); psd =NULL ;
@@ -330,7 +344,7 @@ int psp_assoc_device_get_locked(const struct genl_split_ops *ops,psp_dev_put (psd); }else { - psd =psp_device_get_and_lock (genl_info_net (info), id); + psd =psp_device_get_and_lock (genl_info_net (info), id, false);if (IS_ERR (psd)) { err =PTR_ERR (psd); goto err_sock_put;
@@ -573,7 +587,7 @@ static int psp_nl_stats_get_dumpit_one(struct sk_buff *rsp, struct netlink_callback *cb, struct psp_dev *psd) { - if (psp_dev_check_access(psd, sock_net(rsp->sk))) + if (psp_dev_check_access(psd, sock_net(rsp->sk), false)) return 0; return psp_nl_stats_fill(psd, rsp, genl_info_dump(cb));
Signature · 1 line
2.52.0
PATCH v7 2/5 psp: add new netlink cmd for dev-assoc and dev-disassoc
In reply to Wei WangFrom: Wei Wang <[email protected]>
The main purpose of this cmd is to be able to associate a non-psp-capable device (e.g. veth or netkit) with a psp device. One use case is if we create a pair of veth/netkit, and assign 1 end inside a netns, while leaving the other end within the default netns, with a real PSP device, e.g. netdevsim or a physical PSP-capable NIC. With this command, we could associate the veth/netkit inside the netns with PSP device, so the virtual device could act as PSP-capable device to initiate PSP connections, and performs PSP encryption/decryption on the real PSP device.
--- Documentation/netlink/specs/psp.yaml | 67 +++++- include/net/psp/types.h | 15 ++ include/uapi/linux/psp.h | 13 ++ net/psp/psp-nl-gen.c | 32 +++ net/psp/psp-nl-gen.h | 2 + net/psp/psp_main.c | 21 +- net/psp/psp_nl.c | 309 ++++++++++++++++++++++++++- 7 files changed, 447 insertions(+), 12 deletions(-)
Documentation/netlink/specs/psp.yaml +66 -1
@@ -13,6 +13,17 @@ definitions: hdr0-aes-gmac-128, hdr0-aes-gmac-256] attribute-sets: + - + name: assoc-dev-info + attributes: + - + name: ifindex + doc: ifindex of an associated network device. + type: u32 + - + name: nsid + doc: Network namespace ID of the associated device. + type: s32 - name: dev attributes:
@@ -24,7 +35,9 @@ attribute-sets: min: 1 - name: ifindex - doc: ifindex of the main netdevice linked to the PSP device. + doc: | + ifindex of the main netdevice linked to the PSP device, + or the ifindex to associate with the PSP device. type: u32 - name: psp-versions-cap
@@ -38,6 +51,28 @@ attribute-sets: type: u32 enum: version enum-as-flags: true + - + name: assoc-list + doc: List of associated virtual devices. + type: nest + nested-attributes: assoc-dev-info + multi-attr: true + - + name: nsid + doc: | + Network namespace ID for the device to associate/disassociate. + Optional for dev-assoc and dev-disassoc; if not present, the + device is looked up in the caller's network namespace. + type: s32 + - + name: by-association + doc: | + Flag indicating the PSP device is an associated device from a + different network namespace. + Present when in associated namespace, absent when in primary/host + namespace. + type: flag + - name: assoc attributes:
@@ -170,6 +205,8 @@ operations: - ifindex - psp-versions-cap - psp-versions-ena + - assoc-list + - by-association pre: psp-device-get-locked post: psp-device-unlock dump:
@@ -271,6 +308,34 @@ operations: post: psp-device-unlock dump: reply: *stats-all + - + name: dev-assoc + doc: Associate a network device with a PSP device. + attribute-set: dev + do: + request: + attributes: + - id + - ifindex + - nsid + reply: + attributes: [] + pre: psp-device-get-locked + post: psp-device-unlock + - + name: dev-disassoc + doc: Disassociate a network device from a PSP device. + attribute-set: dev + do: + request: + attributes: + - id + - ifindex + - nsid + reply: + attributes: [] + pre: psp-device-get-locked + post: psp-device-unlock mcast-groups: list:
include/net/psp/types.h +15 -0
@@ -5,6 +5,7 @@#include <linux/mutex.h> #include <linux/refcount.h> +#include <net/net_trackers.h> struct netlink_ext_ack ;
@@ -43,9 +44,22 @@ struct psp_dev_config {u32 versions; }; +/** +* struct psp_assoc_dev - wrapper for associated net_device +* @dev_list: list node for psp_dev::assoc_dev_list +* @assoc_dev: the associated net_device +* @dev_tracker: tracker for the net_device reference +*/ +struct psp_assoc_dev { +struct list_head dev_list; +struct net_device *assoc_dev; +netdevice_tracker dev_tracker; +}; +/** * struct psp_dev - PSP device struct * @main_netdev: original netdevice of this PSP device +* @assoc_dev_list: list of psp_assoc_dev entries associated with this PSP device * @ops: driver callbacks * @caps: device capabilities * @drv_priv: driver priv pointer
@@ -67,6 +81,7 @@ struct psp_dev_config {*/ struct psp_dev {struct net_device *main_netdev; +struct list_head assoc_dev_list;struct psp_dev_ops *ops;struct psp_dev_caps *caps;
include/uapi/linux/psp.h +13 -0
@@ -17,11 +17,22 @@ enum psp_version { PSP_VERSION_HDR0_AES_GMAC_256, }; +enum { +PSP_A_ASSOC_DEV_INFO_IFINDEX =1 , +PSP_A_ASSOC_DEV_INFO_NSID , + + __PSP_A_ASSOC_DEV_INFO_MAX, +PSP_A_ASSOC_DEV_INFO_MAX = (__PSP_A_ASSOC_DEV_INFO_MAX -1 ) +}; +enum {PSP_A_DEV_ID =1 ,PSP_A_DEV_IFINDEX ,PSP_A_DEV_PSP_VERSIONS_CAP ,PSP_A_DEV_PSP_VERSIONS_ENA , +PSP_A_DEV_ASSOC_LIST , +PSP_A_DEV_NSID , +PSP_A_DEV_BY_ASSOCIATION , __PSP_A_DEV_MAX,PSP_A_DEV_MAX = (__PSP_A_DEV_MAX -1 )
@@ -74,6 +85,8 @@ enum {PSP_CMD_RX_ASSOC ,PSP_CMD_TX_ASSOC ,PSP_CMD_GET_STATS , +PSP_CMD_DEV_ASSOC , +PSP_CMD_DEV_DISASSOC , __PSP_CMD_MAX,PSP_CMD_MAX = (__PSP_CMD_MAX -1 )
net/psp/psp-nl-gen.c +32 -0
@@ -53,6 +53,20 @@ static const struct nla_policy psp_get_stats_nl_policy[PSP_A_STATS_DEV_ID + 1] = [PSP_A_STATS_DEV_ID] = NLA_POLICY_MIN(NLA_U32, 1), }; +/* PSP_CMD_DEV_ASSOC - do */ +static const struct nla_policy psp_dev_assoc_nl_policy[PSP_A_DEV_NSID +1 ] = { + [PSP_A_DEV_ID ] =NLA_POLICY_MIN (NLA_U32 ,1 ), + [PSP_A_DEV_IFINDEX ] = { .type =NLA_U32 , }, + [PSP_A_DEV_NSID ] = { .type =NLA_S32 , }, +}; + +/* PSP_CMD_DEV_DISASSOC - do */ +static const struct nla_policy psp_dev_disassoc_nl_policy[PSP_A_DEV_NSID +1 ] = { + [PSP_A_DEV_ID ] =NLA_POLICY_MIN (NLA_U32 ,1 ), + [PSP_A_DEV_IFINDEX ] = { .type =NLA_U32 , }, + [PSP_A_DEV_NSID ] = { .type =NLA_S32 , }, +}; +/* Ops table for psp */ static const struct genl_split_ops psp_nl_ops[] = { {
@@ -119,6 +133,24 @@ static const struct genl_split_ops psp_nl_ops[] = { .dumpit = psp_nl_get_stats_dumpit, .flags = GENL_CMD_CAP_DUMP, }, + { + .cmd = PSP_CMD_DEV_ASSOC, + .pre_doit = psp_device_get_locked, + .doit = psp_nl_dev_assoc_doit, + .post_doit = psp_device_unlock, + .policy = psp_dev_assoc_nl_policy, + .maxattr = PSP_A_DEV_NSID, + .flags = GENL_CMD_CAP_DO, + }, + { + .cmd = PSP_CMD_DEV_DISASSOC, + .pre_doit = psp_device_get_locked, + .doit = psp_nl_dev_disassoc_doit, + .post_doit = psp_device_unlock, + .policy = psp_dev_disassoc_nl_policy, + .maxattr = PSP_A_DEV_NSID, + .flags = GENL_CMD_CAP_DO, + }, }; static const struct genl_multicast_group psp_nl_mcgrps[] = {
net/psp/psp-nl-gen.h +2 -0
@@ -33,6 +33,8 @@ int psp_nl_rx_assoc_doit(struct sk_buff *skb, struct genl_info *info);int psp_nl_tx_assoc_doit (struct sk_buff *skb,struct genl_info *info);int psp_nl_get_stats_doit (struct sk_buff *skb,struct genl_info *info);int psp_nl_get_stats_dumpit (struct sk_buff *skb,struct netlink_callback *cb); +int psp_nl_dev_assoc_doit (struct sk_buff *skb,struct genl_info *info); +int psp_nl_dev_disassoc_doit (struct sk_buff *skb,struct genl_info *info);enum {PSP_NLGRP_MGMT ,
net/psp/psp_main.c +20 -1
@@ -37,8 +37,18 @@ struct mutex psp_devs_lock;*/ int psp_dev_check_access (struct psp_dev *psd,struct net *net,bool admin) { +struct psp_assoc_dev *entry; +if (dev_net (psd->main_netdev) == net)return 0 ; + +if (!admin) { +list_for_each_entry (entry, &psd->assoc_dev_list, dev_list) { +if (dev_net (entry->assoc_dev) == net) +return 0 ; + } + } +return -ENOENT ; }
@@ -74,6 +84,7 @@ psp_dev_create(struct net_device *netdev,return ERR_PTR (-ENOMEM ); psd->main_netdev = netdev; +INIT_LIST_HEAD (&psd->assoc_dev_list); psd->ops = psd_ops; psd->caps = psd_caps; psd->drv_priv = priv_ptr;
@@ -121,6 +132,7 @@ void psp_dev_free(struct psp_dev *psd)*/ void psp_dev_unregister (struct psp_dev *psd) { +struct psp_assoc_dev *entry, *entry_tmp;struct psp_assoc *pas, *next;mutex_lock (&psp_devs_lock);
@@ -140,6 +152,14 @@ void psp_dev_unregister(struct psp_dev *psd)list_for_each_entry_safe (pas, next, &psd->stale_assocs, assocs_list)psp_dev_tx_key_del (psd, pas); +list_for_each_entry_safe (entry, entry_tmp, &psd->assoc_dev_list, + dev_list) { +list_del (&entry->dev_list); +rcu_assign_pointer (entry->assoc_dev->psp_dev,NULL ); +netdev_put (entry->assoc_dev, &entry->dev_tracker); +kfree (entry); + } +rcu_assign_pointer (psd->main_netdev->psp_dev,NULL ); psd->ops =NULL ;
@@ -361,5 +381,4 @@ static int __init psp_init(void)return genl_register_family (&psp_nl_family); } -subsys_initcall (psp_init);
net/psp/psp_nl.c +299 -10
369 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v7 3/5 psp: add a new netdev event for dev unregister
In reply to Wei WangFrom: Wei Wang <[email protected]>
Add a new netdev event for dev unregister and handle the removal of this dev from psp->assoc_dev_list, upon the first successful dev-assoc operation.
--- net/psp/psp.h | 1 + net/psp/psp_main.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++ net/psp/psp_nl.c | 7 +++++ 3 files changed, 78 insertions(+)
net/psp/psp.h +1 -0
@@ -15,6 +15,7 @@ extern struct mutex psp_devs_lock;void psp_dev_free (struct psp_dev *psd);int psp_dev_check_access (struct psp_dev *psd,struct net *net,bool admin); +void psp_attach_netdev_notifier (void );void psp_nl_notify_dev (struct psp_dev *psd,u32 cmd);
net/psp/psp_main.c +70 -0
80 lines not on this page · raw message
net/psp/psp_nl.c +7 -0
@@ -515,6 +515,13 @@ int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info)psp_nl_notify_dev (psd,PSP_CMD_DEV_CHANGE_NTF ); +/* Register netdev notifier for assoc cleanup on success. +* Must drop psd->lock to ensure lock ordering: rtnl_lock -> psd->lock +*/ +mutex_unlock (&psd->lock); +psp_attach_netdev_notifier (); +mutex_lock (&psd->lock); +return psp_nl_reply_send (rsp, info); }
Signature · 1 line
2.52.0
PATCH v7 4/5 selftests/net: Add bpf skb forwarding program
In reply to Wei WangFrom: Wei Wang <[email protected]>
Add nk_redirect.bpf.c, a BPF program that forwards skbs matching some IPv6 prefix received on eth0 ifindex to a specified dev ifindex. bpf_redirect_neigh() is used to make sure neighbor lookup is performed and proper MAC addr is being used.
--- .../drivers/net/hw/nk_redirect.bpf.c | 60 +++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 tools/testing/selftests/drivers/net/hw/nk_redirect.bpf.c
tools/testing/selftests/drivers/net/hw/nk_redirect.bpf.c new file +60 -0
@@ -0,0 +1,60 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* BPF program for redirecting traffic using bpf_redirect_neigh(). +* Unlike bpf_redirect() which preserves L2 headers, bpf_redirect_neigh() +* performs neighbor lookup and fills in the correct L2 addresses for the +* target interface. This is necessary when redirecting across different +* device types (e.g., from netdevsim to netkit). +*/ +#include <linux/bpf.h> +#include <linux/pkt_cls.h> +#include <linux/if_ether.h> +#include <linux/ipv6.h> +#include <linux/in6.h> +#include <bpf/bpf_endian.h> +#include <bpf/bpf_helpers.h> + +#define TC_ACT_OK 0 +#define ETH_P_IPV6 0x86DD + +#define ctx_ptr (field) ((void *)(long)(field)) + +#define v6_p64_equal (a, b) (a.s6_addr32[0] == b.s6_addr32[0] && \ + a.s6_addr32[1] == b.s6_addr32[1]) + +volatile __u32 redirect_ifindex; +volatile __u8 ipv6_prefix[16 ]; + +SEC("tc/ingress") +int tc_redirect (struct __sk_buff *skb) +{ +void *data_end =ctx_ptr (skb->data_end); +void *data =ctx_ptr (skb->data); +struct in6_addr *match_prefix; +struct ipv6hdr *ip6h; +struct ethhdr *eth; + + match_prefix = (struct in6_addr *)ipv6_prefix; + +if (skb->protocol !=bpf_htons (ETH_P_IPV6 )) +return TC_ACT_OK ; + + eth = data; +if ((void *)(eth +1 ) > data_end) +return TC_ACT_OK ; + + ip6h = data +sizeof (struct ethhdr ); +if ((void *)(ip6h +1 ) > data_end) +return TC_ACT_OK ; + +if (!v6_p64_equal (ip6h->daddr, (*match_prefix))) +return TC_ACT_OK ; + +/* +* Use bpf_redirect_neigh() to perform neighbor lookup and fill in +* correct L2 addresses for the target interface. +*/ +return bpf_redirect_neigh (redirect_ifindex,NULL ,0 ,0 ); +} + +char __license[] SEC("license") = "GPL";
Signature · 1 line
2.52.0
PATCH v7 5/5 selftest/net: psp: Add test for dev-assoc/disassoc
In reply to Wei WangFrom: Wei Wang <[email protected]>
Add a new param to NetDrvContEnv to add an additional bpf redirect program on nk_host to redirect traffic to the psp_dev_local. The topology looks like this:
Host NS: psp_dev_local <---> nk_host
| |
| | (netkit pair)
| |
Remote NS: psp_dev_peer Guest NS: nk_guest
(responder) (PSP tests)Add following tests for dev-assoc/dev-disassoc functionality:
1. Test the output of ./tools/net/ynl/pyynl/cli.py --spec Documentation/netlink/specs/psp.yaml --dump dev-get in both default and the guest netns.
2. Test the case where we associate netkit with psp_dev_local, and send PSP traffic from nk_guest to psp_dev_peer in 2 different netns.
3. Test to make sure the key rotation notification is sent to the netns for associated dev as well
4. Test to make sure the dev change notification is sent to the netns for associated dev as well
5. Test for dev-assoc/dev-disassoc without nsid parameter.
6. Test the deletion of nk_guest in client netns, and proper cleanup in the assoc-list for psp dev.
--- tools/testing/selftests/drivers/net/config | 1 + .../selftests/drivers/net/lib/py/env.py | 54 +- tools/testing/selftests/drivers/net/psp.py | 492 ++++++++++++++++-- 3 files changed, 513 insertions(+), 34 deletions(-)
tools/testing/selftests/drivers/net/config +1 -0
@@ -7,4 +7,5 @@ CONFIG_NETCONSOLE=m CONFIG_NETCONSOLE_DYNAMIC=y CONFIG_NETCONSOLE_EXTENDED_LOG=y CONFIG_NETDEVSIM=m +CONFIG_NETKIT=y CONFIG_XDP_SOCKETS=y
tools/testing/selftests/drivers/net/lib/py/env.py +53 -1
82 lines not on this page · raw message
tools/testing/selftests/drivers/net/psp.py +459 -33
541 lines not on this page · raw message
Signature · 1 line
2.52.0
Simon Horman
reviewerRe: [PATCH v7 net-next 1/5] psp: add admin/non-admin version of psp_device_get_locked
In reply to Wei WangThis is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add admin/non-admin version of psp_device_get_locked
This commit introduces two versions of psp_device_get_locked to distinguish between admin operations (dev-set, key-rotation) and non-admin operations (dev-get, rx-assoc, tx-assoc, get-stats).
Quoted 4 lines
Documentation/netlink/specs/psp.yaml +0
-0
[ ... ]
Quoted 4 lines
net/psp/psp-nl-gen.h +0
-0
[ ... ]
Quoted 4 lines
net/psp/psp.h +0
-0
[ ... ]
Quoted 4 lines
net/psp/psp_main.c +0
-0
[ ... ]
Quoted 43 lines
net/psp/psp_nl.c +15
-2@@ -41,7 +41,8 @@ static int psp_nl_reply_send(struct sk_buff *rsp, struct genl_info *info)/* Device stuff */ static struct psp_dev * -psp_device_get_and_lock (struct net *net,struct nlattr *dev_id) +psp_device_get_and_lock (struct net *net,struct nlattr *dev_id, +bool admin) {struct psp_dev *psd;int err;@@ -56,7 +57,7 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)mutex_lock (&psd->lock);mutex_unlock (&psp_devs_lock); - err =psp_dev_check_access (psd, net); + err =psp_dev_check_access (psd, net, admin);if (err) {mutex_unlock (&psd->lock);return ERR_PTR (err);@@ -65,6 +66,18 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)return psd; } +int psp_device_get_locked_admin (const struct genl_split_ops *ops, +struct sk_buff *skb,struct genl_info *info) +{ +if (GENL_REQ_ATTR_CHECK (info,PSP_A_DEV_ID )) +return -EINVAL ; + + info->user_ptr[0 ] =psp_device_get_and_lock (genl_info_net (info), + info->attrs[PSP_A_DEV_ID ], + true); +return PTR_ERR_OR_ZERO (info->user_ptr[0 ]); +} +int psp_device_get_locked (const struct genl_split_ops *ops,struct sk_buff *skb,struct genl_info *info) {
net/psp/psp_nl.c int psp_device_get_locked(const struct genl_split_ops *ops, · line 91
Quoted 8 lines
net/psp/psp_nl.c +2
-1@@ -72,7 +85,8 @@ int psp_device_get_locked(const struct genl_split_ops *ops,return -EINVAL ; info->user_ptr[0 ] =psp_device_get_and_lock (genl_info_net (info), - info->attrs[PSP_A_DEV_ID ]); + info->attrs[PSP_A_DEV_ID ], + false);return PTR_ERR_OR_ZERO (info->user_ptr[0 ]); }
This isn't a bug, but psp_device_get_locked_admin() and psp_device_get_locked() are identical except for the true/false parameter passed to psp_device_get_and_lock(). Would it be cleaner to introduce a common helper like __psp_device_get_locked(bool admin) to avoid the duplication?
Reference: https://lore.kernel.org/netdev/[email protected]/
Daniel Zahka raised this refactoring suggestion in v2, noting the two functions are entirely identical modulo the admin parameter.
Quoted 9 lines
net/psp/psp_nl.c +1
-1@@ -160,7 +174,7 @@ static int psp_nl_dev_get_dumpit_one(struct sk_buff *rsp, struct netlink_callback *cb, struct psp_dev *psd) { - if (psp_dev_check_access(psd, sock_net(rsp->sk))) + if (psp_dev_check_access(psd, sock_net(rsp->sk), false)) return 0; return psp_nl_dev_fill(psd, rsp, genl_info_dump(cb));
[ ... ]
Simon Horman
reviewerRe: [PATCH v7 net-next 2/5] psp: add new netlink cmd for dev-assoc and dev-disassoc
In reply to Wei WangThis is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add new netlink cmd for dev-assoc and dev-disassoc
This commit adds new netlink commands (PSP_CMD_DEV_ASSOC and PSP_CMD_DEV_DISASSOC) to associate non-PSP-capable devices (e.g. veth or netkit) with a PSP device, enabling them to act as PSP-capable devices by performing PSP encryption/decryption on the real PSP device.
Quoted 12 lines
net/psp/psp_main.c +2
-0@@ -37,8 +37,18 @@ struct mutex psp_devs_lock;*/ int psp_dev_check_access (struct psp_dev *psd,struct net *net,bool admin) { +struct psp_assoc_dev *entry; +if (dev_net (psd->main_netdev) == net)return 0 ;
net/psp/psp_main.c struct mutex psp_devs_lock; · line 53
Quoted 10 lines
net/psp/psp_main.c +8
-0+ +if (!admin) { +list_for_each_entry (entry, &psd->assoc_dev_list, dev_list) { +if (dev_net (entry->assoc_dev) == net) +return 0 ; + } + } +return -ENOENT ; }
Does this list iteration have a race with psp_dev_unregister()?
The new list_for_each_entry iteration over psd->assoc_dev_list in psp_dev_check_access is called from psp_assoc_device_get_locked (net/psp/psp_nl.c:615) without holding psd->lock.
That caller obtains psd via psp_dev_get_for_sock which takes a refcount but does not acquire psd->lock before calling psp_dev_check_access(psd, ..., false). Since admin is false, the new code path is entered.
Concurrently, psp_dev_unregister holds psd->lock and performs list_del + kfree on the same assoc_dev_list entries:
net/psp/psp_main.c:psp_dev_unregister() {
...
list_for_each_entry_safe(entry, entry_tmp, &psd->assoc_dev_list,
dev_list) {
list_del(&entry->dev_list);
rcu_assign_pointer(entry->assoc_dev->psp_dev, NULL);
netdev_put(entry->assoc_dev, &entry->dev_tracker);
kfree(entry);
}
...
}The list iterator in psp_dev_check_access can follow a next pointer into freed memory, causing a use-after-free.
Other callers (psp_device_get_and_lock, psp_nl_dev_get_dumpit, psp_nl_get_stats_dumpit) correctly hold psd->lock before calling this function, so the race is specific to the psp_assoc_device_get_locked path.
Quoted 4 lines
net/psp/psp_nl.c +0
-0
[ ... ]
Quoted 58 lines
net/psp/psp_nl.c +54
-0@@ -291,6 +446,140 @@ int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info)return err; } +int psp_nl_dev_assoc_doit (struct sk_buff *skb,struct genl_info *info) +{ +struct psp_dev *psd = info->user_ptr[0 ]; +struct psp_assoc_dev *psp_assoc_dev; +struct net_device *assoc_dev; +u32 assoc_ifindex; +struct sk_buff *rsp; +struct net *net; +int nsid; + +if (GENL_REQ_ATTR_CHECK (info,PSP_A_DEV_IFINDEX )) +return -EINVAL ; + +if (info->attrs[PSP_A_DEV_NSID ]) { + nsid =nla_get_s32 (info->attrs[PSP_A_DEV_NSID ]); + + net =get_net_ns_by_id (genl_info_net (info), nsid); +if (!net) { +NL_SET_BAD_ATTR (info->extack, + info->attrs[PSP_A_DEV_NSID ]); +return -EINVAL ; + } + }else { + net =get_net (genl_info_net (info)); + } + + psp_assoc_dev =kzalloc (sizeof (*psp_assoc_dev),GFP_KERNEL ); +if (!psp_assoc_dev) { +put_net (net); +return -ENOMEM ; + } + + assoc_ifindex =nla_get_u32 (info->attrs[PSP_A_DEV_IFINDEX ]); + assoc_dev =netdev_get_by_index (net, assoc_ifindex, + &psp_assoc_dev->dev_tracker, +GFP_KERNEL ); +if (!assoc_dev) { +put_net (net); +kfree (psp_assoc_dev); +NL_SET_BAD_ATTR (info->extack, info->attrs[PSP_A_DEV_IFINDEX ]); +return -ENODEV ; + } + +/* Check if device is already associated with a PSP device */ +if (rcu_access_pointer (assoc_dev->psp_dev)) { +NL_SET_ERR_MSG (info->extack, +"Device already associated with a PSP device" ); +netdev_put (assoc_dev, &psp_assoc_dev->dev_tracker); +put_net (net); +kfree (psp_assoc_dev); +return -EBUSY ; + } + + psp_assoc_dev->assoc_dev = assoc_dev;
net/psp/psp_nl.c int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info) · line 512
Quoted 10 lines
net/psp/psp_nl.c +10
-0+ rsp =psp_nl_reply_new (info); +if (!rsp) { +netdev_put (assoc_dev, &psp_assoc_dev->dev_tracker); +put_net (net); +kfree (psp_assoc_dev); +return -ENOMEM ; + } + +rcu_assign_pointer (assoc_dev->psp_dev, psd); +list_add_tail (&psp_assoc_dev->dev_list, &psd->assoc_dev_list);
Can two concurrent PSP_CMD_DEV_ASSOC operations for the same net_device but different PSP devices both succeed?
The function checks rcu_access_pointer(assoc_dev->psp_dev) to verify the target net_device is not already associated with a PSP device, then later writes rcu_assign_pointer(assoc_dev->psp_dev, psd) and adds the wrapper to psd->assoc_dev_list.
These operations are only serialized by psd->lock, which is a per-PSP-device mutex acquired by the pre_doit callback psp_device_get_locked(). With parallel_ops=true in the psp_nl_family (net/psp/psp-nl-gen.c:165), two concurrent PSP_CMD_DEV_ASSOC operations targeting different PSP devices (each holding their own distinct psd->lock) but specifying the same target ifindex can both pass the rcu_access_pointer() NULL check.
Both then call rcu_assign_pointer(assoc_dev->psp_dev, psd) and list_add_tail(), resulting in the same net_device being added to two different PSP devices' assoc_dev_list. The second rcu_assign_pointer silently overwrites the first.
This leaves the first PSP device's assoc_dev_list containing an entry whose net_device->psp_dev points to the wrong PSP device:
Thread A (PSP device psd_A) Thread B (PSP device psd_B)
================================ ================================
mutex_lock(&psd_A->lock) mutex_lock(&psd_B->lock)
rcu_access_pointer(dev->psp_dev)
-> NULL, proceed rcu_access_pointer(dev->psp_dev)
-> NULL, proceed
rcu_assign_pointer(dev->psp_dev, rcu_assign_pointer(dev->psp_dev,
psd_A) psd_B)
list_add_tail(entry_A, list_add_tail(entry_B,
&psd_A->list) &psd_B->list)A subsequent disassociation from the first PSP device would find its stale psp_assoc_dev entry, call rcu_assign_pointer(found->assoc_dev->psp_dev, NULL), and clear the pointer that now belongs to the second PSP device, corrupting cross-device state.
The data path check in psp_validate_xmit (rcu_access_pointer(dev->psp_dev) == pas->psd) can then fail incorrectly.
Simon Horman
reviewerRe: [PATCH v7 net-next 3/5] psp: add a new netdev event for dev unregister
In reply to Wei WangThis is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add a new netdev event for dev unregister
This commit adds a netdev notifier to handle device unregistration events and clean up PSP device associations when the associated network device is removed.
Quoted 16 lines
net/psp/psp.h +1
-0@@ -15,6 +15,7 @@ extern struct mutex psp_devs_lock;void psp_dev_free (struct psp_dev *psd);int psp_dev_check_access (struct psp_dev *psd,struct net *net,bool admin); +void psp_attach_netdev_notifier (void );void psp_nl_notify_dev (struct psp_dev *psd,u32 cmd);net/psp/psp_main.c +0
-0
[ ... ]
Quoted 10 lines
net/psp/psp_main.c +10
-0+/** +* psp_attach_netdev_notifier() - register netdev notifier on first use +* +* Register the netdevice notifier when the first device association +* is created. In many installations no associations will be created and +* the notifier won't be needed. +* +* Must be called without psd->lock held, due to lock ordering: +* rtnl_lock -> psd->lock (the notifier callback runs under rtnl_lock +* and takes psd->lock).
net/psp/psp_main.c int psp_dev_rcv(struct sk_buff *skb, u16 dev_id, u8 generation, bool strip_icv) · line 442
Quoted 10 lines
net/psp/psp_main.c +10
-0+*/ +void psp_attach_netdev_notifier (void ) +{ +if (READ_ONCE (psp_notifier_registered)) +return ; + +mutex_lock (&psp_devs_lock); +if (!psp_notifier_registered) { +register_netdevice_notifier (&psp_netdev_notifier); +WRITE_ONCE (psp_notifier_registered, true);
^^^^
What happens if register_netdevice_notifier() fails? The function ignores the return value and sets psp_notifier_registered to true anyway.
register_netdevice_notifier() can fail with errors like -ENOMEM or errors from replaying existing device registrations. If registration fails, psp_notifier_registered will be permanently set to true, preventing any future registration attempts.
This means the notifier will never be registered, so NETDEV_UNREGISTER events will be silently missed and PSP device associations will never be cleaned up on device removal.
Quoted 7 lines
net/psp/psp_main.c +4
-0+ } +mutex_unlock (&psp_devs_lock); +} +static int __init psp_init (void ) {mutex_init (&psp_devs_lock);
[ ... ]
Quoted 5 lines
net/psp/psp_nl.c +0
-0
net/psp/psp_nl.c int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info) · line 524
Quoted 10 lines
net/psp/psp_nl.c +7
-0@@ -515,6 +515,13 @@ int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info)psp_nl_notify_dev (psd,PSP_CMD_DEV_CHANGE_NTF ); +/* Register netdev notifier for assoc cleanup on success. +* Must drop psd->lock to ensure lock ordering: rtnl_lock -> psd->lock +*/ +mutex_unlock (&psd->lock); +psp_attach_netdev_notifier (); +mutex_lock (&psd->lock); +
Can dropping psd->lock here create a use-after-free window?
The pre_doit callback psp_device_get_and_lock() obtains psd via xa_load() and holds psd->lock, but does NOT take a refcount on psd.
When psd->lock is dropped here, psp_dev_unregister() can acquire both psp_devs_lock and psd->lock, complete teardown, and call psp_dev_put() which drops the refcount to 0:
net/psp/psp_main.c:psp_dev_unregister() {
...
mutex_lock(&psp_devs_lock);
...
mutex_lock(&psd->lock);
...
xa_erase(&psp_devs, psd->id);
...
mutex_unlock(&psd->lock);
mutex_unlock(&psp_devs_lock);
psp_dev_put(psd); // can trigger psp_dev_free()
}This would trigger psp_dev_free() which calls mutex_destroy(&psd->lock) and kfree_rcu(psd, rcu).
When this function then calls mutex_lock(&psd->lock), it operates on a destroyed mutex and logically freed memory. The post_doit callback psp_device_unlock() would then call mutex_unlock(&psd->lock) on the same destroyed mutex.
Should a refcount be taken on psd before dropping the lock, or should the notifier registration be moved outside the doit handler?
Quoted 2 lines
net/psp/psp_nl.c +0
-0return psp_nl_reply_send (rsp, info); }
Re: [PATCH v7 net-next 1/5] psp: add admin/non-admin version of psp_device_get_locked
In reply to Simon HormanQuoted 98 lines · Simon Horman
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add admin/non-admin version ofpsp_device_get_lockedThis commit introduces two versions of
psp_device_get_lockedto distinguish between admin operations (dev-set, key-rotation) and non-admin operations (dev-get, rx-assoc, tx-assoc, get-stats).Quoted 4 lines
Documentation/netlink/specs/psp.yaml +0
-0[ ... ]
Quoted 4 lines
net/psp/psp-nl-gen.h +0
-0[ ... ]
Quoted 4 lines
net/psp/psp.h +0
-0[ ... ]
Quoted 4 lines
net/psp/psp_main.c +0
-0[ ... ]
Quoted 43 lines
net/psp/psp_nl.c +15
-2@@ -41,7 +41,8 @@ static int psp_nl_reply_send(struct sk_buff *rsp, struct genl_info *info)/* Device stuff */ static struct psp_dev * -psp_device_get_and_lock (struct net *net,struct nlattr *dev_id) +psp_device_get_and_lock (struct net *net,struct nlattr *dev_id, +bool admin) {struct psp_dev *psd;int err;@@ -56,7 +57,7 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)mutex_lock (&psd->lock);mutex_unlock (&psp_devs_lock); - err =psp_dev_check_access (psd, net); + err =psp_dev_check_access (psd, net, admin);if (err) {mutex_unlock (&psd->lock);return ERR_PTR (err);@@ -65,6 +66,18 @@ psp_device_get_and_lock(struct net *net, struct nlattr *dev_id)return psd; } +int psp_device_get_locked_admin (const struct genl_split_ops *ops, +struct sk_buff *skb,struct genl_info *info) +{ +if (GENL_REQ_ATTR_CHECK (info,PSP_A_DEV_ID )) +return -EINVAL ; + + info->user_ptr[0 ] =psp_device_get_and_lock (genl_info_net (info), + info->attrs[PSP_A_DEV_ID ], + true); +return PTR_ERR_OR_ZERO (info->user_ptr[0 ]); +} +int psp_device_get_locked (const struct genl_split_ops *ops,struct sk_buff *skb,struct genl_info *info) {net/psp/psp_nl.c int psp_device_get_locked(const struct genl_split_ops *ops, · line 91
Quoted 8 lines
net/psp/psp_nl.c +2
-1@@ -72,7 +85,8 @@ int psp_device_get_locked(const struct genl_split_ops *ops,return -EINVAL ; info->user_ptr[0 ] =psp_device_get_and_lock (genl_info_net (info), - info->attrs[PSP_A_DEV_ID ]); + info->attrs[PSP_A_DEV_ID ], + false);return PTR_ERR_OR_ZERO (info->user_ptr[0 ]); }This isn't a bug, but
psp_device_get_locked_admin()andpsp_device_get_locked()are identical except for the true/false parameter passed topsp_device_get_and_lock(). Would it be cleaner to introduce a common helper like__psp_device_get_locked(bool admin)to avoid the duplication?
Quoted 3 lines
Reference: https://lore.kernel.org/netdev/[email protected]/
Daniel Zahka raised this refactoring suggestion in v2, noting the two functions are entirely identical modulo the admin parameter.
Ack. Addressing in the next version.
Quoted 12 lines
Quoted 10 lines
net/psp/psp_nl.c +1
-1@@ -160,7 +174,7 @@ static int psp_nl_dev_get_dumpit_one(struct sk_buff *rsp, struct netlink_callback *cb, struct psp_dev *psd) { - if (psp_dev_check_access(psd, sock_net(rsp->sk))) + if (psp_dev_check_access(psd, sock_net(rsp->sk), false)) return 0; return psp_nl_dev_fill(psd, rsp, genl_info_dump(cb));[ ... ]
Re: [PATCH v7 net-next 2/5] psp: add new netlink cmd for dev-assoc and dev-disassoc
In reply to Simon HormanQuoted 61 lines · Simon Horman
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add new netlink cmd for dev-assoc and dev-disassocThis commit adds new netlink commands (
PSP_CMD_DEV_ASSOCandPSP_CMD_DEV_DISASSOC) to associate non-PSP-capable devices (e.g. veth or netkit) with a PSP device, enabling them to act as PSP-capable devices by performing PSP encryption/decryption on the real PSP device.Quoted 12 lines
net/psp/psp_main.c +2
-0@@ -37,8 +37,18 @@ struct mutex psp_devs_lock;*/ int psp_dev_check_access (struct psp_dev *psd,struct net *net,bool admin) { +struct psp_assoc_dev *entry; +if (dev_net (psd->main_netdev) == net)return 0 ;net/psp/psp_main.c struct mutex psp_devs_lock; · line 53
Quoted 10 lines
net/psp/psp_main.c +8
-0+ +if (!admin) { +list_for_each_entry (entry, &psd->assoc_dev_list, dev_list) { +if (dev_net (entry->assoc_dev) == net) +return 0 ; + } + } +return -ENOENT ; }Does this list iteration have a race with
psp_dev_unregister()?The new
list_for_each_entryiteration overpsd->assoc_dev_listinpsp_dev_check_accessis called frompsp_assoc_device_get_locked(net/psp/psp_nl.c:615) without holdingpsd->lock.That caller obtains psd via
psp_dev_get_for_sockwhich takes a refcount but does not acquirepsd->lockbefore callingpsp_dev_check_access(psd, ..., false). Since admin is false, the new code path is entered.Concurrently,
psp_dev_unregisterholdspsd->lockand performslist_del+ kfree on the sameassoc_dev_listentries:net/psp/psp_main.c:psp_dev_unregister() { ... list_for_each_entry_safe(entry, entry_tmp, &psd->assoc_dev_list, dev_list) { list_del(&entry->dev_list); rcu_assign_pointer(entry->assoc_dev->psp_dev, NULL); netdev_put(entry->assoc_dev, &entry->dev_tracker); kfree(entry); } ... }
Quoted 7 lines
The list iterator in
psp_dev_check_accesscan follow a next pointer into freed memory, causing a use-after-free.Other callers (
psp_device_get_and_lock,psp_nl_dev_get_dumpit,psp_nl_get_stats_dumpit) correctly holdpsd->lockbefore calling this function, so the race is specific to thepsp_assoc_device_get_lockedpath.
Grabbing psd->lock before psp_dev_check_access () in the next versions.
Quoted 110 lines
Quoted 4 lines
net/psp/psp_nl.c +0
-0[ ... ]
Quoted 58 lines
net/psp/psp_nl.c +54
-0@@ -291,6 +446,140 @@ int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info)return err; } +int psp_nl_dev_assoc_doit (struct sk_buff *skb,struct genl_info *info) +{ +struct psp_dev *psd = info->user_ptr[0 ]; +struct psp_assoc_dev *psp_assoc_dev; +struct net_device *assoc_dev; +u32 assoc_ifindex; +struct sk_buff *rsp; +struct net *net; +int nsid; + +if (GENL_REQ_ATTR_CHECK (info,PSP_A_DEV_IFINDEX )) +return -EINVAL ; + +if (info->attrs[PSP_A_DEV_NSID ]) { + nsid =nla_get_s32 (info->attrs[PSP_A_DEV_NSID ]); + + net =get_net_ns_by_id (genl_info_net (info), nsid); +if (!net) { +NL_SET_BAD_ATTR (info->extack, + info->attrs[PSP_A_DEV_NSID ]); +return -EINVAL ; + } + }else { + net =get_net (genl_info_net (info)); + } + + psp_assoc_dev =kzalloc (sizeof (*psp_assoc_dev),GFP_KERNEL ); +if (!psp_assoc_dev) { +put_net (net); +return -ENOMEM ; + } + + assoc_ifindex =nla_get_u32 (info->attrs[PSP_A_DEV_IFINDEX ]); + assoc_dev =netdev_get_by_index (net, assoc_ifindex, + &psp_assoc_dev->dev_tracker, +GFP_KERNEL ); +if (!assoc_dev) { +put_net (net); +kfree (psp_assoc_dev); +NL_SET_BAD_ATTR (info->extack, info->attrs[PSP_A_DEV_IFINDEX ]); +return -ENODEV ; + } + +/* Check if device is already associated with a PSP device */ +if (rcu_access_pointer (assoc_dev->psp_dev)) { +NL_SET_ERR_MSG (info->extack, +"Device already associated with a PSP device" ); +netdev_put (assoc_dev, &psp_assoc_dev->dev_tracker); +put_net (net); +kfree (psp_assoc_dev); +return -EBUSY ; + } + + psp_assoc_dev->assoc_dev = assoc_dev;net/psp/psp_nl.c int psp_nl_key_rotate_doit(struct sk_buff *skb, struct genl_info *info) · line 512
Quoted 10 lines
net/psp/psp_nl.c +10
-0+ rsp =psp_nl_reply_new (info); +if (!rsp) { +netdev_put (assoc_dev, &psp_assoc_dev->dev_tracker); +put_net (net); +kfree (psp_assoc_dev); +return -ENOMEM ; + } + +rcu_assign_pointer (assoc_dev->psp_dev, psd); +list_add_tail (&psp_assoc_dev->dev_list, &psd->assoc_dev_list);Can two concurrent
PSP_CMD_DEV_ASSOCoperations for the samenet_devicebut different PSP devices both succeed?The function checks
rcu_access_pointer(assoc_dev->psp_dev)to verify the targetnet_deviceis not already associated with a PSP device, then later writesrcu_assign_pointer(assoc_dev->psp_dev, psd)and adds the wrapper topsd->assoc_dev_list.These operations are only serialized by
psd->lock, which is a per-PSP-device mutex acquired by thepre_doitcallbackpsp_device_get_locked(). With parallel_ops=true in thepsp_nl_family(net/psp/psp-nl-gen.c:165), two concurrentPSP_CMD_DEV_ASSOCoperations targeting different PSP devices (each holding their own distinctpsd->lock) but specifying the same target ifindex can both pass thercu_access_pointer()NULL check.Both then call
rcu_assign_pointer(assoc_dev->psp_dev, psd)andlist_add_tail(), resulting in the samenet_devicebeing added to two different PSP devices'assoc_dev_list. The secondrcu_assign_pointersilently overwrites the first.This leaves the first PSP device's
assoc_dev_listcontaining an entry whosenet_device->psp_devpoints to the wrong PSP device:Thread A (PSP device psd_A) Thread B (PSP device psd_B) ================================ ================================ mutex_lock(&psd_A->lock) mutex_lock(&psd_B->lock) rcu_access_pointer(dev->psp_dev) -> NULL, proceed rcu_access_pointer(dev->psp_dev) -> NULL, proceed rcu_assign_pointer(dev->psp_dev, rcu_assign_pointer(dev->psp_dev, psd_A) psd_B) list_add_tail(entry_A, list_add_tail(entry_B, &psd_A->list) &psd_B->list)
Quoted 7 lines
A subsequent disassociation from the first PSP device would find its stale
psp_assoc_deventry, callrcu_assign_pointer(found->assoc_dev->psp_dev, NULL), and clear the pointer that now belongs to the second PSP device, corrupting cross-device state.The data path check in
psp_validate_xmit(rcu_access_pointer(dev->psp_dev)==pas->psd) can then fail incorrectly.
Changing to use cmpxchg() on assoc_dev->psp_dev in the next version.
Re: [PATCH v7 net-next 3/5] psp: add a new netdev event for dev unregister
In reply to Simon HormanQuoted 56 lines · Simon Horman
This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
psp: add a new netdev event for dev unregisterThis commit adds a netdev notifier to handle device unregistration events and clean up PSP device associations when the associated network device is removed.
Quoted 16 lines
net/psp/psp.h +1
-0@@ -15,6 +15,7 @@ extern struct mutex psp_devs_lock;void psp_dev_free (struct psp_dev *psd);int psp_dev_check_access (struct psp_dev *psd,struct net *net,bool admin); +void psp_attach_netdev_notifier (void );void psp_nl_notify_dev (struct psp_dev *psd,u32 cmd);net/psp/psp_main.c +0
-0[ ... ]
Quoted 10 lines
net/psp/psp_main.c +10
-0+/** +* psp_attach_netdev_notifier() - register netdev notifier on first use +* +* Register the netdevice notifier when the first device association +* is created. In many installations no associations will be created and +* the notifier won't be needed. +* +* Must be called without psd->lock held, due to lock ordering: +* rtnl_lock -> psd->lock (the notifier callback runs under rtnl_lock +* and takes psd->lock). net/psp/psp_main.c int psp_dev_rcv(struct sk_buff *skb, u16 dev_id, u8 generation, bool strip_icv) · line 442
Quoted 10 lines
net/psp/psp_main.c +10
-0+*/ +void psp_attach_netdev_notifier (void ) +{ +if (READ_ONCE (psp_notifier_registered)) +return ; + +mutex_lock (&psp_devs_lock); +if (!psp_notifier_registered) { +register_netdevice_notifier (&psp_netdev_notifier); +WRITE_ONCE (psp_notifier_registered, true);^^^^What happens if
register_netdevice_notifier()fails? The function ignores the return value and setspsp_notifier_registeredto true anyway.
Quoted 8 lines
register_netdevice_notifier()can fail with errors like-ENOMEMor errors from replaying existing device registrations. If registration fails,psp_notifier_registeredwill be permanently set to true, preventing any future registration attempts.This means the notifier will never be registered, so
NETDEV_UNREGISTERevents will be silently missed and PSP device associations will never be cleaned up on device removal.
Adding the failing check in the next version.
Quoted 53 lines
Quoted 8 lines
net/psp/psp_main.c +4
-0+ } +mutex_unlock (&psp_devs_lock); +} +static int __init psp_init (void ) {mutex_init (&psp_devs_lock);[ ... ]
Quoted 5 lines
net/psp/psp_nl.c +0
-0net/psp/psp_nl.c int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info) · line 524
Quoted 10 lines
net/psp/psp_nl.c +7
-0@@ -515,6 +515,13 @@ int psp_nl_dev_assoc_doit(struct sk_buff *skb, struct genl_info *info)psp_nl_notify_dev (psd,PSP_CMD_DEV_CHANGE_NTF ); +/* Register netdev notifier for assoc cleanup on success. +* Must drop psd->lock to ensure lock ordering: rtnl_lock -> psd->lock +*/ +mutex_unlock (&psd->lock); +psp_attach_netdev_notifier (); +mutex_lock (&psd->lock); +Can dropping
psd->lockhere create a use-after-free window?The
pre_doitcallbackpsp_device_get_and_lock()obtains psd viaxa_load()and holdspsd->lock, but does NOT take a refcount on psd.When
psd->lockis dropped here,psp_dev_unregister()can acquire bothpsp_devs_lockandpsd->lock, complete teardown, and callpsp_dev_put()which drops the refcount to 0:net/psp/psp_main.c:psp_dev_unregister() { ... mutex_lock(&psp_devs_lock); ... mutex_lock(&psd->lock); ... xa_erase(&psp_devs, psd->id); ... mutex_unlock(&psd->lock); mutex_unlock(&psp_devs_lock); psp_dev_put(psd); // can trigger psp_dev_free() }This would trigger
psp_dev_free()which callsmutex_destroy(&psd->lock)andkfree_rcu(psd, rcu).
Quoted 8 lines
When this function then calls
mutex_lock(&psd->lock), it operates on a destroyed mutex and logically freed memory. Thepost_doitcallbackpsp_device_unlock()would then callmutex_unlock(&psd->lock)on the same destroyed mutex.Should a refcount be taken on psd before dropping the lock, or should the notifier registration be moved outside the doit handler?
Will move psp_attach_netdev_notifier() to pre_doit handler.
Quoted 2 lines
Quoted 2 lines
net/psp/psp_nl.c +0
-0return psp_nl_reply_send (rsp, info); }