hv_netvsc: fix potential deadlock in netvsc_vf_setxdp()
Subsystems (6)
PATCH v2 1/1 hv_netvsc: fix potential deadlock in netvsc_vf_setxdp()
The MANA driver's probe registers netdevice via the following call chain:
mana_probe()
register_netdev()
register_netdevice()
register_netdevice() calls notifier callback for netvsc driver, holding the netdev mutex via netdev_lock_ops().
Further this netvsc notifier callback end up attempting to acquire the same lock again in dev_xdp_propagate() leading to deadlock.
netvsc_netdev_event()
netvsc_vf_setxdp()
dev_xdp_propagate()
This deadlock was not observed so far because net_shaper_ops was never set, and thus the lock was effectively a no-op in this case. Fix this by using netif_xdp_propagate() instead of dev_xdp_propagate() to avoid recursive locking in this path.
Also, clean up the unregistration path by removing the unnecessary call to netvsc_vf_setxdp(), since unregister_netdevice_many_notify() already performs this cleanup via dev_xdp_uninstall().
---
[V2]
- Modified commit message
drivers/net/hyperv/netvsc_bpf.c | 2 +- drivers/net/hyperv/netvsc_drv.c | 2 -- net/core/dev.c | 1 + 3 files changed, 2 insertions(+), 3 deletions(-)
drivers/net/hyperv/netvsc_bpf.c +1 -1
@@ -183,7 +183,7 @@ int netvsc_vf_setxdp(struct net_device *vf_netdev, struct bpf_prog *prog) xdp.command =XDP_SETUP_PROG ; xdp.prog = prog; - ret =dev_xdp_propagate (vf_netdev, &xdp); + ret =netif_xdp_propagate (vf_netdev, &xdp);if (ret && prog)bpf_prog_put (prog);
drivers/net/hyperv/netvsc_drv.c +0 -2
@@ -2462,8 +2462,6 @@ static int netvsc_unregister_vf(struct net_device *vf_netdev)netdev_info (ndev,"VF unregistering: %s\n" , vf_netdev->name); -netvsc_vf_setxdp (vf_netdev,NULL ); -reinit_completion (&net_device_ctx->vf_add);netdev_rx_handler_unregister (vf_netdev);netdev_upper_dev_unlink (vf_netdev, ndev);
net/core/dev.c +1 -0
@@ -9953,6 +9953,7 @@ int netif_xdp_propagate(struct net_device *dev, struct netdev_bpf *bpf)return dev->netdev_ops->ndo_bpf (dev, bpf); } +EXPORT_SYMBOL_GPL (netif_xdp_propagate);u32 dev_xdp_prog_id (struct net_device *dev,enum bpf_xdp_mode mode) {
Signature · 1 line
2.43.0
Quoted 25 lines · Saurabh Sengar
The MANA driver's probe registers netdevice via the following call chain:
mana_probe()
register_netdev()
register_netdevice()
register_netdevice()calls notifier callback for netvsc driver, holding the netdev mutex vianetdev_lock_ops().Further this netvsc notifier callback end up attempting to acquire the same lock again in
dev_xdp_propagate()leading to deadlock.
netvsc_netdev_event()
netvsc_vf_setxdp()
dev_xdp_propagate()This deadlock was not observed so far because
net_shaper_opswas never set, and thus the lock was effectively a no-op in this case. Fix this by usingnetif_xdp_propagate()instead ofdev_xdp_propagate()to avoid recursive locking in this path.Also, clean up the unregistration path by removing the unnecessary call to
netvsc_vf_setxdp(), sinceunregister_netdevice_many_notify()already performs this cleanup viadev_xdp_uninstall().
Quoted 5 lines
Fixes 97246d6d21c2 ("net: hold netdev instance lock during ndo_bpf")Signed-off-by Saurabh Sengar <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>
Thanks,
Sundeep
Quoted 50 lines
---
[V2]
- Modified commit messagedrivers/net/hyperv/netvsc_bpf.c | 2 +- drivers/net/hyperv/netvsc_drv.c | 2 -- net/core/dev.c | 1 + 3 files changed, 2 insertions(+), 3 deletions(-)drivers/net/hyperv/netvsc_bpf.c +1
-1@@ -183,7 +183,7 @@ int netvsc_vf_setxdp(struct net_device *vf_netdev, struct bpf_prog *prog) xdp.command =XDP_SETUP_PROG ; xdp.prog = prog; - ret =dev_xdp_propagate (vf_netdev, &xdp); + ret =netif_xdp_propagate (vf_netdev, &xdp);if (ret && prog)bpf_prog_put (prog);drivers/net/hyperv/netvsc_drv.c +0
-2@@ -2462,8 +2462,6 @@ static int netvsc_unregister_vf(struct net_device *vf_netdev)netdev_info (ndev,"VF unregistering: %s\n" , vf_netdev->name); -netvsc_vf_setxdp (vf_netdev,NULL ); -reinit_completion (&net_device_ctx->vf_add);netdev_rx_handler_unregister (vf_netdev);netdev_upper_dev_unlink (vf_netdev, ndev);net/core/dev.c +1
-0@@ -9953,6 +9953,7 @@ int netif_xdp_propagate(struct net_device *dev, struct netdev_bpf *bpf)return dev->netdev_ops->ndo_bpf (dev, bpf); } +EXPORT_SYMBOL_GPL (netif_xdp_propagate);u32 dev_xdp_prog_id (struct net_device *dev,enum bpf_xdp_mode mode) {Signature · 1 line
2.43.0
Jakub Kicinski
maintainerQuoted 9 lines · On Wed, 21 May 2025 03:25:03 -0700 Saurabh Sengar wrote:
The MANA driver's probe registers netdevice via the following call chain:
mana_probe()
register_netdev()
register_netdevice()
register_netdevice()calls notifier callback for netvsc driver, holding the netdev mutex vianetdev_lock_ops().
Quoted 8 lines
Further this netvsc notifier callback end up attempting to acquire the same lock again in
dev_xdp_propagate()leading to deadlock.
netvsc_netdev_event()
netvsc_vf_setxdp()
dev_xdp_propagate()This deadlock was not observed so far because
net_shaper_opswas never set,
The lock is on the VF, I think you meant to say that no device you use in Azure is ops locked?
There's also the call to netvsc_register_vf() on probe path, please fix or explain why it doesn't need locking in the commit message.
Signature · 1 line
pw-bot: cr
Quoted 17 lines · Jakub Kicinski
Quoted 16 lines · On Wed, 21 May 2025 03:25:03 -0700 Saurabh Sengar wrote:
The MANA driver's probe registers netdevice via the following call chain:
mana_probe()
register_netdev()
register_netdevice()
register_netdevice()calls notifier callback for netvsc driver, holding the netdev mutex vianetdev_lock_ops().Further this netvsc notifier callback end up attempting to acquire the same lock again in
dev_xdp_propagate()leading to deadlock.
netvsc_netdev_event()
netvsc_vf_setxdp()
dev_xdp_propagate()
Quoted 7 lines
Quoted 1 line
This deadlock was not observed so far because
net_shaper_opswas never set,The lock is on the VF, I think you meant to say that no device you use in Azure is ops locked?
There's also the call to
netvsc_register_vf()on probe path, please fix or explain why it doesn't need locking in the commit message.
This patch specifically addresses the netvsc_register_vf() path only. I omitted the mention of netvsc_register_vf() in the commit message to keep the function path shorter. The full stack trace is provided below:
[ 92.542180] dev_xdp_propagate+0x2c/0x1b0 [ 92.542185] netvsc_vf_setxdp+0x10d/0x180 [hv_netvsc] [ 92.542192] netvsc_register_vf.part.0+0x179/0x200 [hv_netvsc] [ 92.542196] netvsc_netdev_event+0x267/0x340 [hv_netvsc] [ 92.542200] notifier_call_chain+0x5f/0xc0 [ 92.542203] raw_notifier_call_chain+0x16/0x20 [ 92.542205] call_netdevice_notifiers_info+0x52/0xa0 [ 92.542209] register_netdevice+0x7c8/0xaa0 [ 92.542211] register_netdev+0x1f/0x40 [ 92.542214] mana_probe+0x6e2/0x8e0 [mana] [ 92.542220] mana_gd_probe+0x187/0x220 [mana]
If you prefer I can update the stack trace in commit meesage From:
netvsc_netdev_event()
netvsc_vf_setxdp()
dev_xdp_propagate()
To:
netvsc_netdev_event()
netvsc_register_vf()
netvsc_vf_setxdp()
dev_xdp_propagate()
- Saurabh
Quoted 2 lines
Signature · 1 line
pw-bot: cr
Quoted 13 lines · Jakub Kicinski
Quoted 12 lines · On Wed, 21 May 2025 03:25:03 -0700 Saurabh Sengar wrote:
The MANA driver's probe registers netdevice via the following call chain:
mana_probe()
register_netdev()
register_netdevice()
register_netdevice()calls notifier callback for netvsc driver, holding the netdev mutex vianetdev_lock_ops().Further this netvsc notifier callback end up attempting to acquire the same lock again in
dev_xdp_propagate()leading to deadlock.
Quoted 8 lines
Quoted 5 lines
netvsc_netdev_event()
netvsc_vf_setxdp()
dev_xdp_propagate()This deadlock was not observed so far because
net_shaper_opswas never set,The lock is on the VF, I think you meant to say that no device you use in Azure is ops locked?
That's right.
Quoted 3 lines
There's also the call to
netvsc_register_vf()on probe path, please fix or explain why it doesn't need locking in the commit message.
On rethinking I realize you were referring to the netvsc_probe() path not mana_probe(). Since this lock is effectively a no-op, it doesn't really matter whether it's there or not.
However, I think we can revisit this when we add ops for any of the VFs.
- Saurabh