virtio-net: Link queues to NAPIs
Subsystems (5)
v5 cover virtio-net: Link queues to NAPIs
Greetings:
Welcome to v5. Patches 1, 2, and 4 have no functional changes only updated tags. Patch 3 was refactored as requested by Jason. See the changelog below and the commit message for details.
Jakub recently commented [1] that I should not hold this series on virtio-net linking queues to NAPIs behind other important work that is on-going and suggested I re-spin, so here we are :)
As per the discussion on the v3 [2], now both RX and TX NAPIs use the API to link queues to NAPIs. Since TX-only NAPIs don't have a NAPI ID, commit 6597e8d35851 ("netdev-genl: Elide napi_id when not present") now correctly elides the TX-only NAPIs (instead of printing zero) when the queues and NAPIs are linked.
As per the discussion on the v4 [3], patch 3 has been refactored to hold RTNL only in the specific locations which need it as Jason requested.
See the commit message of patch 3 for an example of how to get the NAPI to queue mapping information.
See the commit message of patch 4 for an example of how NAPI IDs are persistent despite queue count changes.
Thanks,
Joe
[1]: https://lore.kernel.org/netdev/[email protected]/
[2]: https://lore.kernel.org/netdev/[email protected]/
[3]: https://lore.kernel.org/netdev/CACGkMEv=ejJnOWDnAu7eULLvrqXjkMkTL4cbi-uCTUhCpKN_GA@mail.gmail.com/
v5:
- Patch 1 added Acked-by's from Michael and Jason. Added Tested-by from Lei. No functional changes.
- Patch 2 added Acked-by's from Michael and Jason. Added Tested-by from Lei. No functional changes.
- Patch 3:
- Refactored as Jason requested, eliminating the virtnet_queue_set_napi helper entirely, and explicitly holding RTNL in the 3 locations where needed (refill_work, freeze, and restore).
- Commit message updated to outline the known paths at the time the commit was written.
- Patch 4 added Acked-by from Michael. Added Tested-by from Lei. No functional changes.
v4: https://lore.kernel.org/lkml/[email protected]/
- Dropped Jakub's patch (previously patch 1).
- Significant refactor from v3 affecting patches 1-3.
- Patch 4 added tags from Jason and Gerhard.
rfcv3: https://lore.kernel.org/netdev/[email protected]/
- patch 3:
- Removed the xdp checks completely, as Gerhard Engleder pointed out, they are likely not necessary.
- patch 4:
- Added Xuan Zhuo's Reviewed-by.
v2: https://lore.kernel.org/netdev/[email protected]/
- patch 1:
- New in the v2 from Jakub.
- patch 2:
- Previously patch 1, unchanged from v1.
- Added Gerhard Engleder's Reviewed-by.
- Added Lei Yang's Tested-by.
- patch 3:
- Introduced virtnet_napi_disable to eliminate duplicated code in virtnet_xdp_set, virtnet_rx_pause, virtnet_disable_queue_pair, refill_work as suggested by Jason Wang.
- As a result of the above refactor, dropped Reviewed-by and Tested-by from patch 3.
- patch 4:
- New in v2. Adds persistent NAPI configuration. See commit message for more details.
v1: https://lore.kernel.org/netdev/[email protected]/
Joe Damato (4):
virtio-net: Refactor napi_enable paths
virtio-net: Refactor napi_disable paths
virtio-net: Map NAPIs to queues
virtio_net: Use persistent NAPI config
drivers/net/virtio_net.c | 95 ++++++++++++++++++++++++++++------------ 1 file changed, 67 insertions(+), 28 deletions(-)
base-commit: 7fe0353606d77a32c4c7f2814833dd1c043ebdd2
Signature · 1 line
2.45.2
PATCH v5 1/4 virtio-net: Refactor napi_enable paths
Refactor virtnet_napi_enable and virtnet_napi_tx_enable to take a struct receive_queue. Create a helper, virtnet_napi_do_enable, which contains the logic to enable a NAPI.
--- drivers/net/virtio_net.c | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-)
drivers/net/virtio_net.c +21 -16
@@ -2807,7 +2807,8 @@ static void skb_recv_done(struct virtqueue *rvq)virtqueue_napi_schedule (&rq->napi, rvq); } -static void virtnet_napi_enable (struct virtqueue *vq,struct napi_struct *napi) +static void virtnet_napi_do_enable (struct virtqueue *vq, +struct napi_struct *napi) {napi_enable (napi);
@@ -2820,10 +2821,16 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi)local_bh_enable (); } -static void virtnet_napi_tx_enable (struct virtnet_info *vi, -struct virtqueue *vq, -struct napi_struct *napi) +static void virtnet_napi_enable (struct receive_queue *rq) { +virtnet_napi_do_enable (rq->vq, &rq->napi); +} + +static void virtnet_napi_tx_enable (struct send_queue *sq) +{ +struct virtnet_info *vi = sq->vq->vdev->priv; +struct napi_struct *napi = &sq->napi; +if (!napi->weight)return ;
@@ -2835,7 +2842,7 @@ static void virtnet_napi_tx_enable(struct virtnet_info *vi,return ; } -return virtnet_napi_enable (vq, napi); +virtnet_napi_do_enable (sq->vq, napi); }static void virtnet_napi_tx_disable (struct napi_struct *napi)
@@ -2856,7 +2863,7 @@ static void refill_work(struct work_struct *work)napi_disable (&rq->napi); still_empty = !try_fill_recv (vi, rq,GFP_KERNEL ); -virtnet_napi_enable (rq->vq, &rq->napi); +virtnet_napi_enable (rq);/* In theory, this can happen: if we don't get any buffers in * we will *never* try to fill again.
@@ -3073,8 +3080,8 @@ static int virtnet_enable_queue_pair(struct virtnet_info *vi, int qp_index)if (err <0 ) goto err_xdp_reg_mem_model; -virtnet_napi_enable (vi->rq[qp_index].vq, &vi->rq[qp_index].napi); -virtnet_napi_tx_enable (vi, vi->sq[qp_index].vq, &vi->sq[qp_index].napi); +virtnet_napi_enable (&vi->rq[qp_index]); +virtnet_napi_tx_enable (&vi->sq[qp_index]);return 0 ;
@@ -3339,7 +3346,7 @@ static void virtnet_rx_resume(struct virtnet_info *vi, struct receive_queue *rq)schedule_delayed_work (&vi->refill,0 );if (running) -virtnet_napi_enable (rq->vq, &rq->napi); +virtnet_napi_enable (rq); } static int virtnet_rx_resize(struct virtnet_info *vi,
@@ -3402,7 +3409,7 @@ static void virtnet_tx_resume(struct virtnet_info *vi, struct send_queue *sq)__netif_tx_unlock_bh (txq);if (running) -virtnet_napi_tx_enable (vi, sq->vq, &sq->napi); +virtnet_napi_tx_enable (sq); } static int virtnet_tx_resize(struct virtnet_info *vi, struct send_queue *sq,
@@ -5983,9 +5990,8 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog,if (old_prog)bpf_prog_put (old_prog);if (netif_running (dev)) { -virtnet_napi_enable (vi->rq[i].vq, &vi->rq[i].napi); -virtnet_napi_tx_enable (vi, vi->sq[i].vq, - &vi->sq[i].napi); +virtnet_napi_enable (&vi->rq[i]); +virtnet_napi_tx_enable (&vi->sq[i]); } }
@@ -6000,9 +6006,8 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog,if (netif_running (dev)) {for (i =0 ; i < vi->max_queue_pairs; i++) { -virtnet_napi_enable (vi->rq[i].vq, &vi->rq[i].napi); -virtnet_napi_tx_enable (vi, vi->sq[i].vq, - &vi->sq[i].napi); +virtnet_napi_enable (&vi->rq[i]); +virtnet_napi_tx_enable (&vi->sq[i]); } }if (prog)
Signature · 1 line
2.45.2
PATCH v5 2/4 virtio-net: Refactor napi_disable paths
In reply to Joe DamatoCreate virtnet_napi_disable helper and refactor virtnet_napi_tx_disable to take a struct send_queue.
--- drivers/net/virtio_net.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-)
drivers/net/virtio_net.c +17 -8
@@ -2845,12 +2845,21 @@ static void virtnet_napi_tx_enable(struct send_queue *sq)virtnet_napi_do_enable (sq->vq, napi); } -static void virtnet_napi_tx_disable (struct napi_struct *napi) +static void virtnet_napi_tx_disable (struct send_queue *sq) { +struct napi_struct *napi = &sq->napi; +if (napi->weight)napi_disable (napi); } +static void virtnet_napi_disable (struct receive_queue *rq) +{ +struct napi_struct *napi = &rq->napi; + +napi_disable (napi); +} +static void refill_work (struct work_struct *work) { struct virtnet_info *vi =
@@ -2861,7 +2870,7 @@ static void refill_work(struct work_struct *work)for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i]; -napi_disable (&rq->napi); +virtnet_napi_disable (rq); still_empty = !try_fill_recv (vi, rq,GFP_KERNEL );virtnet_napi_enable (rq);
@@ -3060,8 +3069,8 @@ static int virtnet_poll(struct napi_struct *napi, int budget)static void virtnet_disable_queue_pair (struct virtnet_info *vi,int qp_index) { -virtnet_napi_tx_disable (&vi->sq[qp_index].napi); -napi_disable (&vi->rq[qp_index].napi); +virtnet_napi_tx_disable (&vi->sq[qp_index]); +virtnet_napi_disable (&vi->rq[qp_index]);xdp_rxq_info_unreg (&vi->rq[qp_index].xdp_rxq); }
@@ -3333,7 +3342,7 @@ static void virtnet_rx_pause(struct virtnet_info *vi, struct receive_queue *rq)bool running =netif_running (vi->dev);if (running) { -napi_disable (&rq->napi); +virtnet_napi_disable (rq);virtnet_cancel_dim (vi, &rq->dim); } }
@@ -3375,7 +3384,7 @@ static void virtnet_tx_pause(struct virtnet_info *vi, struct send_queue *sq) qindex = sq - vi->sq;if (running) -virtnet_napi_tx_disable (&sq->napi); +virtnet_napi_tx_disable (sq); txq =netdev_get_tx_queue (vi->dev, qindex);
@@ -5952,8 +5961,8 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog,/* Make sure NAPI is not using any XDP TX queues for RX. */ if (netif_running (dev)) {for (i =0 ; i < vi->max_queue_pairs; i++) { -napi_disable (&vi->rq[i].napi); -virtnet_napi_tx_disable (&vi->sq[i].napi); +virtnet_napi_disable (&vi->rq[i]); +virtnet_napi_tx_disable (&vi->sq[i]); } }
Signature · 1 line
2.45.2
PATCH v5 3/4 virtio-net: Map NAPIs to queues
In reply to Joe DamatoUse netif_queue_set_napi to map NAPIs to queue IDs so that the mapping can be accessed by user apps. Note that the netif_queue_set_napi currently requires RTNL, so care must be taken to ensure RTNL is held on paths where this API might be reached.
The paths in the driver where this API can be reached appear to be:
- ndo_open, ndo_close, which hold RTNL so no driver change is needed.
- rx_pause, rx_resume, tx_pause, tx_resume are reached either via an ethtool ioctl or via XSK - neither path requires a driver change.
- power management paths (which call open and close), which have been updated to hold/release RTNL.
- refill_work, which has been updated to hold RTNL.
$ ethtool -i ens4 | grep driver driver: virtio_net
$ sudo ethtool -L ens4 combined 4
$ ./tools/net/ynl/pyynl/cli.py \
--spec Documentation/netlink/specs/netdev.yaml \
--dump queue-get --json='{"ifindex": 2}'
[{'id': 0, 'ifindex': 2, 'napi-id': 8289, 'type': 'rx'},
{'id': 1, 'ifindex': 2, 'napi-id': 8290, 'type': 'rx'},
{'id': 2, 'ifindex': 2, 'napi-id': 8291, 'type': 'rx'},
{'id': 3, 'ifindex': 2, 'napi-id': 8292, 'type': 'rx'},
{'id': 0, 'ifindex': 2, 'type': 'tx'},
{'id': 1, 'ifindex': 2, 'type': 'tx'},
{'id': 2, 'ifindex': 2, 'type': 'tx'},
{'id': 3, 'ifindex': 2, 'type': 'tx'}]Note that virtio_net has TX-only NAPIs which do not have NAPI IDs, so the lack of 'napi-id' in the above output is expected.
--- drivers/net/virtio_net.c | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-)
drivers/net/virtio_net.c +26 -2
@@ -2823,13 +2823,18 @@ static void virtnet_napi_do_enable(struct virtqueue *vq,static void virtnet_napi_enable (struct receive_queue *rq) { +struct virtnet_info *vi = rq->vq->vdev->priv; +int qidx =vq2rxq (rq->vq); +virtnet_napi_do_enable (rq->vq, &rq->napi); +netif_queue_set_napi (vi->dev, qidx,NETDEV_QUEUE_TYPE_RX , &rq->napi); }static void virtnet_napi_tx_enable (struct send_queue *sq) {struct virtnet_info *vi = sq->vq->vdev->priv;struct napi_struct *napi = &sq->napi; +int qidx =vq2txq (sq->vq);if (!napi->weight)return ;
@@ -2843,20 +2848,28 @@ static void virtnet_napi_tx_enable(struct send_queue *sq) }virtnet_napi_do_enable (sq->vq, napi); +netif_queue_set_napi (vi->dev, qidx,NETDEV_QUEUE_TYPE_TX , napi); }static void virtnet_napi_tx_disable (struct send_queue *sq) { +struct virtnet_info *vi = sq->vq->vdev->priv;struct napi_struct *napi = &sq->napi; +int qidx =vq2txq (sq->vq); -if (napi->weight) +if (napi->weight) { +netif_queue_set_napi (vi->dev, qidx,NETDEV_QUEUE_TYPE_TX ,NULL );napi_disable (napi); + } }static void virtnet_napi_disable (struct receive_queue *rq) { +struct virtnet_info *vi = rq->vq->vdev->priv;struct napi_struct *napi = &rq->napi; +int qidx =vq2rxq (rq->vq); +netif_queue_set_napi (vi->dev, qidx,NETDEV_QUEUE_TYPE_RX ,NULL );napi_disable (napi); }
@@ -2870,9 +2883,15 @@ static void refill_work(struct work_struct *work)for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i]; +rtnl_lock ();virtnet_napi_disable (rq); +rtnl_unlock (); + still_empty = !try_fill_recv (vi, rq,GFP_KERNEL ); + +rtnl_lock ();virtnet_napi_enable (rq); +rtnl_unlock ();/* In theory, this can happen: if we don't get any buffers in * we will *never* try to fill again.
@@ -5650,8 +5669,11 @@ static void virtnet_freeze_down(struct virtio_device *vdev)netif_tx_lock_bh (vi->dev);netif_device_detach (vi->dev);netif_tx_unlock_bh (vi->dev); -if (netif_running (vi->dev)) +if (netif_running (vi->dev)) { +rtnl_lock ();virtnet_close (vi->dev); +rtnl_unlock (); + } }static int init_vqs (struct virtnet_info *vi);
@@ -5671,7 +5693,9 @@ static int virtnet_restore_up(struct virtio_device *vdev)enable_rx_mode_work (vi);if (netif_running (vi->dev)) { +rtnl_lock (); err =virtnet_open (vi->dev); +rtnl_unlock ();if (err)return err; }
Signature · 1 line
2.45.2
Re: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 32 lines · Joe Damato
Use
netif_queue_set_napito map NAPIs to queue IDs so that the mapping can be accessed by user apps. Note that thenetif_queue_set_napicurrently requires RTNL, so care must be taken to ensure RTNL is held on paths where this API might be reached.The paths in the driver where this API can be reached appear to be:
-
ndo_open,ndo_close, which hold RTNL so no driver change is needed.
-rx_pause,rx_resume,tx_pause,tx_resumeare reached either via an ethtool ioctl or via XSK - neither path requires a driver change.
- power management paths (which call open and close), which have been updated to hold/release RTNL.
-refill_work, which has been updated to hold RTNL.$ ethtool -i ens4 | grep driver driver: virtio_net$ sudo ethtool -L ens4 combined 4$ ./tools/net/ynl/pyynl/cli.py \ --spec Documentation/netlink/specs/netdev.yaml \ --dump queue-get --json='{"ifindex": 2}' [{'id': 0, 'ifindex': 2, 'napi-id': 8289, 'type': 'rx'}, {'id': 1, 'ifindex': 2, 'napi-id': 8290, 'type': 'rx'}, {'id': 2, 'ifindex': 2, 'napi-id': 8291, 'type': 'rx'}, {'id': 3, 'ifindex': 2, 'napi-id': 8292, 'type': 'rx'}, {'id': 0, 'ifindex': 2, 'type': 'tx'}, {'id': 1, 'ifindex': 2, 'type': 'tx'}, {'id': 2, 'ifindex': 2, 'type': 'tx'}, {'id': 3, 'ifindex': 2, 'type': 'tx'}]
Quoted 5 lines
Note that
virtio_nethas TX-only NAPIs which do not have NAPI IDs, so the lack of 'napi-id' in the above output is expected.Signed-off-by Joe Damato <[email protected]>---
Thanks
Jakub Kicinski
maintainerRe: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
In reply to Joe Damatodrivers/net/virtio_net.c static void refill_work(struct work_struct *work) · line 2894
Quoted 13 lines · On Thu, 27 Feb 2025 18:50:13 +0000 Joe Damato wrote:
drivers/net/virtio_net.c +6
-0@@ -2870,9 +2883,15 @@ static void refill_work(struct work_struct *work)for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i]; +rtnl_lock ();virtnet_napi_disable (rq); +rtnl_unlock (); + still_empty = !try_fill_recv (vi, rq,GFP_KERNEL ); + +rtnl_lock ();virtnet_napi_enable (rq); +rtnl_unlock ();
Looks to me like refill_work is cancelled _sync while holding rtnl_lock from the close path. I think this could deadlock?
Signature · 1 line
pw-bot: cr
Re: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Thu, 27 Feb 2025 18:50:13 +0000 Joe Damato wrote:
(quoted diff) +0
-0@@ -2870,9 +2883,15 @@ static void refill_work(struct work_struct *work) for (i = 0; i < vi->curr_queue_pairs; i++) { struct receive_queue *rq = &vi->rq[i];
Quoted 12 lines
Review static void refill_work(struct work_struct *work) · line 2894
Quoted 9 lines
(quoted diff) +6
-0+ rtnl_lock(); virtnet_napi_disable(rq); + rtnl_unlock(); + still_empty = !try_fill_recv(vi, rq, GFP_KERNEL); + + rtnl_lock(); virtnet_napi_enable(rq); + rtnl_unlock();Looks to me like
refill_workis cancelled_syncwhile holdingrtnl_lockfrom the close path. I think this could deadlock?
Good catch, thank you!
It looks like this is also the case in the failure path on virtnet_open.
Jason: do you have any suggestions?
It looks like in both open and close disable_delayed_refill is called first, before the cancel_delayed_work_sync.
Would something like this solve the problem?
drivers/net/virtio_net.c +7 -0
@@ -2880,6 +2880,13 @@ static void refill_work(struct work_struct *work)bool still_empty;int i; +spin_lock (&vi->refill_lock); +if (!vi->refill_enabled) { +spin_unlock (&vi->refill_lock); +return ; + } +spin_unlock (&vi->refill_lock); +for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i];
Re: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 39 lines · Joe Damato
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Thu, 27 Feb 2025 18:50:13 +0000 Joe Damato wrote:
drivers/net/virtio_net.c +0
-0@@ -2870,9 +2883,15 @@ static void refill_work(struct work_struct *work)for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i];Quoted 12 lines
Review static void refill_work(struct work_struct *work) · line 2894
Quoted 9 lines
(quoted diff) +6
-0+ rtnl_lock(); virtnet_napi_disable(rq); + rtnl_unlock(); + still_empty = !try_fill_recv(vi, rq, GFP_KERNEL); + + rtnl_lock(); virtnet_napi_enable(rq); + rtnl_unlock();Looks to me like
refill_workis cancelled_syncwhile holdingrtnl_lockfrom the close path. I think this could deadlock?Good catch, thank you!
It looks like this is also the case in the failure path on
virtnet_open.Jason: do you have any suggestions?
It looks like in both open and close
disable_delayed_refillis called first, before thecancel_delayed_work_sync.Would something like this solve the problem?
drivers/net/virtio_net.c +0
-0@@ -2880,6 +2880,13 @@ static void refill_work(struct work_struct *work)bool still_empty;int i;
drivers/net/virtio_net.c static void refill_work(struct work_struct *work) · line 2892
Quoted 10 lines
drivers/net/virtio_net.c +7
-0+spin_lock (&vi->refill_lock); +if (!vi->refill_enabled) { +spin_unlock (&vi->refill_lock); +return ; + } +spin_unlock (&vi->refill_lock); +for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i];
Err, I suppose this also doesn't work because:
CPU0 CPU1
rtnl_lock (before CPU0 calls disable_delayed_refill)
virtnet_close refill_work
rtnl_lock() cancel_sync <= deadlock
Need to give this a bit more thought.
Re: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 53 lines · Joe Damato
Quoted 39 lines · Joe Damato
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Thu, 27 Feb 2025 18:50:13 +0000 Joe Damato wrote:
(quoted diff) +0
-0@@ -2870,9 +2883,15 @@ static void refill_work(struct work_struct *work) for (i = 0; i < vi->curr_queue_pairs; i++) { struct receive_queue *rq = &vi->rq[i];Quoted 12 lines
Review static void refill_work(struct work_struct *work) · line 2894
Quoted 9 lines
(quoted diff) +6
-0+ rtnl_lock(); virtnet_napi_disable(rq); + rtnl_unlock(); + still_empty = !try_fill_recv(vi, rq, GFP_KERNEL); + + rtnl_lock(); virtnet_napi_enable(rq); + rtnl_unlock();Looks to me like
refill_workis cancelled_syncwhile holdingrtnl_lockfrom the close path. I think this could deadlock?Good catch, thank you!
It looks like this is also the case in the failure path on
virtnet_open.Jason: do you have any suggestions?
It looks like in both open and close
disable_delayed_refillis called first, before thecancel_delayed_work_sync.Would something like this solve the problem?
drivers/net/virtio_net.c +0
-0@@ -2880,6 +2880,13 @@ static void refill_work(struct work_struct *work)bool still_empty;int i;drivers/net/virtio_net.c static void refill_work(struct work_struct *work) · line 2892
Quoted 10 lines
drivers/net/virtio_net.c +7
-0+spin_lock (&vi->refill_lock); +if (!vi->refill_enabled) { +spin_unlock (&vi->refill_lock); +return ; + } +spin_unlock (&vi->refill_lock); +for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i];Err, I suppose this also doesn't work because:
Quoted 7 lines
CPU0 CPU1 rtnl_lock (before CPU0 calls disable_delayed_refill) virtnet_close refill_work rtnl_lock()
cancel_sync<= deadlockNeed to give this a bit more thought.
How about we don't use the API at all from refill_work?
Patch 4 adds consistent NAPI config state and refill_work isn't a queue resize maybe we don't need to call the netif_queue_set_napi at all since the NAPI IDs are persisted in the NAPI config state and refill_work shouldn't change that?
In which case, we could go back to what refill_work was doing before and avoid the problem entirely.
What do you think ?
drivers/net/virtio_net.c +2 -8
@@ -2883,15 +2883,9 @@ static void refill_work(struct work_struct *work)for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i]; -rtnl_lock (); -virtnet_napi_disable (rq); -rtnl_unlock (); - +napi_disable (&rq->napi); still_empty = !try_fill_recv (vi, rq,GFP_KERNEL ); - -rtnl_lock (); -virtnet_napi_enable (rq); -rtnl_unlock (); +virtnet_napi_do_enable (rq->vq, &rq->napi);/* In theory, this can happen: if we don't get any buffers in * we will *never* try to fill again.
Jakub Kicinski
maintainerRe: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 32 lines · On Mon, 3 Mar 2025 13:33:10 -0500 Joe Damato wrote:
Quoted 17 lines
Quoted 4 lines
(quoted diff) +0
-0@@ -2880,6 +2880,13 @@ static void refill_work(struct work_struct *work) bool still_empty; int i;Review static void refill_work(struct work_struct *work) · line 2892
Quoted 10 lines
(quoted diff) +7
-0+ spin_lock(&vi->refill_lock); + if (!vi->refill_enabled) { + spin_unlock(&vi->refill_lock); + return; + } + spin_unlock(&vi->refill_lock); + for (i = 0; i < vi->curr_queue_pairs; i++) { struct receive_queue *rq = &vi->rq[i];Err, I suppose this also doesn't work because:
Quoted 7 lines
CPU0 CPU1 rtnl_lock (before CPU0 calls disable_delayed_refill) virtnet_close refill_work rtnl_lock()
cancel_sync<= deadlockNeed to give this a bit more thought.
How about we don't use the API at all from
refill_work?Patch 4 adds consistent NAPI config state and
refill_workisn't a queue resize maybe we don't need to call thenetif_queue_set_napiat all since the NAPI IDs are persisted in the NAPI config state andrefill_workshouldn't change that?
Quoted 4 lines
In which case, we could go back to what
refill_workwas doing before and avoid the problem entirely.What do you think ?
Should work, I think. Tho, I suspect someone will want to add queue API support to virtio sooner or later, and they will run into the same problem with the netdev instance lock, as all of ndo_close() will then be covered with netdev->lock.
More thorough and idiomatic way to solve the problem would be to cancel the work non-sync in ndo_close, add cancel with _sync after netdev is unregistered (in virtnet_remove()) when the lock is no longer held, then wrap the entire work with a relevant lock and check if netif_running() to return early in case of a race.
Middle ground would be to do what you suggested above and just leave a well worded comment somewhere that will show up in diffs adding queue API support?
Re: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 43 lines · Jakub Kicinski
Quoted 32 lines · On Mon, 3 Mar 2025 13:33:10 -0500 Joe Damato wrote:
Quoted 17 lines
Quoted 4 lines
(quoted diff) +0
-0@@ -2880,6 +2880,13 @@ static void refill_work(struct work_struct *work) bool still_empty; int i;Review static void refill_work(struct work_struct *work) · line 2892
Quoted 10 lines
(quoted diff) +7
-0+ spin_lock(&vi->refill_lock); + if (!vi->refill_enabled) { + spin_unlock(&vi->refill_lock); + return; + } + spin_unlock(&vi->refill_lock); + for (i = 0; i < vi->curr_queue_pairs; i++) { struct receive_queue *rq = &vi->rq[i];Err, I suppose this also doesn't work because:
Quoted 7 lines
CPU0 CPU1 rtnl_lock (before CPU0 calls disable_delayed_refill) virtnet_close refill_work rtnl_lock()
cancel_sync<= deadlockNeed to give this a bit more thought.
How about we don't use the API at all from
refill_work?Patch 4 adds consistent NAPI config state and
refill_workisn't a queue resize maybe we don't need to call thenetif_queue_set_napiat all since the NAPI IDs are persisted in the NAPI config state andrefill_workshouldn't change that?Quoted 4 lines
In which case, we could go back to what
refill_workwas doing before and avoid the problem entirely.What do you think ?
Should work, I think. Tho, I suspect someone will want to add queue API support to virtio sooner or later, and they will run into the same problem with the netdev instance lock, as all of
ndo_close()will then be covered withnetdev->lock.
Quoted 5 lines
More thorough and idiomatic way to solve the problem would be to cancel the work non-sync in
ndo_close, add cancel with_syncafter netdev is unregistered (invirtnet_remove()) when the lock is no longer held, then wrap the entire work with a relevant lock and check ifnetif_running()to return early in case of a race.
Thanks for the guidance. I am happy to make an attempt at implementing this in a future, separate series that follows this one (probably after netdev conf in a few weeks :).
Quoted 3 lines
Middle ground would be to do what you suggested above and just leave a well worded comment somewhere that will show up in diffs adding queue API support?
Jason, Michael, et. al.: what do you think ? I don't want to spin up a v6 if you are opposed to proceeding this way. Please let me know.
Re: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 55 lines · Joe Damato
Quoted 43 lines · Jakub Kicinski
Quoted 32 lines · On Mon, 3 Mar 2025 13:33:10 -0500 Joe Damato wrote:
Quoted 17 lines
Quoted 4 lines
(quoted diff) +0
-0@@ -2880,6 +2880,13 @@ static void refill_work(struct work_struct *work) bool still_empty; int i;Review static void refill_work(struct work_struct *work) · line 2892
Quoted 10 lines
(quoted diff) +7
-0+ spin_lock(&vi->refill_lock); + if (!vi->refill_enabled) { + spin_unlock(&vi->refill_lock); + return; + } + spin_unlock(&vi->refill_lock); + for (i = 0; i < vi->curr_queue_pairs; i++) { struct receive_queue *rq = &vi->rq[i];Err, I suppose this also doesn't work because:
Quoted 7 lines
CPU0 CPU1 rtnl_lock (before CPU0 calls disable_delayed_refill) virtnet_close refill_work rtnl_lock()
cancel_sync<= deadlockNeed to give this a bit more thought.
How about we don't use the API at all from
refill_work?Patch 4 adds consistent NAPI config state and
refill_workisn't a queue resize maybe we don't need to call thenetif_queue_set_napiat all since the NAPI IDs are persisted in the NAPI config state andrefill_workshouldn't change that?Quoted 4 lines
In which case, we could go back to what
refill_workwas doing before and avoid the problem entirely.What do you think ?
Should work, I think. Tho, I suspect someone will want to add queue API support to virtio sooner or later, and they will run into the same problem with the netdev instance lock, as all of
ndo_close()will then be covered withnetdev->lock.Quoted 5 lines
More thorough and idiomatic way to solve the problem would be to cancel the work non-sync in
ndo_close, add cancel with_syncafter netdev is unregistered (invirtnet_remove()) when the lock is no longer held, then wrap the entire work with a relevant lock and check ifnetif_running()to return early in case of a race.Thanks for the guidance. I am happy to make an attempt at implementing this in a future, separate series that follows this one (probably after netdev conf in a few weeks :).
Quoted 8 lines
Quoted 3 lines
Middle ground would be to do what you suggested above and just leave a well worded comment somewhere that will show up in diffs adding queue API support?
Jason, Michael, et. al.: what do you think ? I don't want to spin up a v6 if you are opposed to proceeding this way. Please let me know.
Maybe, but need to make sure there's no use-after-free (etc. virtnet_close() has several callers).
Thanks
Re: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 4 lines · Jason Wang
Quoted 3 lines · Joe Damato
Quoted 1 line · Jakub Kicinski
On Mon, 3 Mar 2025 13:33:10 -0500 Joe Damato wrote:
[...]
Quoted 11 lines
Quoted 8 lines
Quoted 3 lines
Middle ground would be to do what you suggested above and just leave a well worded comment somewhere that will show up in diffs adding queue API support?
Jason, Michael, et. al.: what do you think ? I don't want to spin up a v6 if you are opposed to proceeding this way. Please let me know.
Maybe, but need to make sure there's no use-after-free (etc.
virtnet_close()has several callers).
Sorry, I think I am missing something. Can you say more?
I was asking: if I add the following diff below to patch 3, will that be acceptable for you as a middle ground until a more idiomatic implementation can be done ?
Since this diff leaves refill_work as it functioned before, it avoids the problem Jakub pointed out and shouldn't introduce any bugs since refill_work isn't changing from the original implementation ?
drivers/net/virtio_net.c +2 -8
@@ -2883,15 +2883,9 @@ static void refill_work(struct work_struct *work)for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i]; -rtnl_lock (); -virtnet_napi_disable (rq); -rtnl_unlock (); - +napi_disable (&rq->napi); still_empty = !try_fill_recv (vi, rq,GFP_KERNEL ); - -rtnl_lock (); -virtnet_napi_enable (rq); -rtnl_unlock (); +virtnet_napi_do_enable (rq->vq, &rq->napi);/* In theory, this can happen: if we don't get any buffers in * we will *never* try to fill again.
Re: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 18 lines · Joe Damato
Quoted 4 lines · Jason Wang
Quoted 3 lines · Joe Damato
Quoted 1 line · Jakub Kicinski
On Mon, 3 Mar 2025 13:33:10 -0500 Joe Damato wrote:
[...]
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines
Middle ground would be to do what you suggested above and just leave a well worded comment somewhere that will show up in diffs adding queue API support?
Jason, Michael, et. al.: what do you think ? I don't want to spin up a v6 if you are opposed to proceeding this way. Please let me know.
Quoted 8 lines
Quoted 2 lines
Maybe, but need to make sure there's no use-after-free (etc.
virtnet_close()has several callers).Sorry, I think I am missing something. Can you say more?
I was asking: if I add the following diff below to patch 3, will that be acceptable for you as a middle ground until a more idiomatic implementation can be done ?
Yes, I misunderstand you before.
Quoted 19 lines
Since this diff leaves
refill_workas it functioned before, it avoids the problem Jakub pointed out and shouldn't introduce any bugs sincerefill_workisn't changing from the original implementation ?drivers/net/virtio_net.c +0
-4@@ -2883,15 +2883,9 @@ static void refill_work(struct work_struct *work)for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i]; -rtnl_lock (); -virtnet_napi_disable (rq); -rtnl_unlock (); -
drivers/net/virtio_net.c static void refill_work(struct work_struct *work) · line 2891
Quoted 10 lines
drivers/net/virtio_net.c +2
-4+napi_disable (&rq->napi); still_empty = !try_fill_recv (vi, rq,GFP_KERNEL ); - -rtnl_lock (); -virtnet_napi_enable (rq); -rtnl_unlock (); +virtnet_napi_do_enable (rq->vq, &rq->napi);/* In theory, this can happen: if we don't get any buffers in * we will *never* try to fill again.
Works for me.
Thanks
Re: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
In reply to Jakub KicinskiQuoted 25 lines · Jakub Kicinski
Quoted 8 lines · On Mon, 3 Mar 2025 13:33:10 -0500 Joe Damato wrote:
How about we don't use the API at all from
refill_work?Patch 4 adds consistent NAPI config state and
refill_workisn't a queue resize maybe we don't need to call thenetif_queue_set_napiat all since the NAPI IDs are persisted in the NAPI config state andrefill_workshouldn't change that?Quoted 4 lines
In which case, we could go back to what
refill_workwas doing before and avoid the problem entirely.What do you think ?
Should work, I think. Tho, I suspect someone will want to add queue API support to virtio sooner or later, and they will run into the same problem with the netdev instance lock, as all of
ndo_close()will then be covered withnetdev->lock.More thorough and idiomatic way to solve the problem would be to cancel the work non-sync in
ndo_close, add cancel with_syncafter netdev is unregistered (invirtnet_remove()) when the lock is no longer held, then wrap the entire work with a relevant lock and check ifnetif_running()to return early in case of a race.
Quoted 3 lines
Middle ground would be to do what you suggested above and just leave a well worded comment somewhere that will show up in diffs adding queue API support?
Seems like Jason agrees that leaving refill_work unmodified will work [1].
I think leaving a comment is a good idea and am happy to do so. Not sure where would be a good spot for it.
Two spots that come to mind are:
- in virtnet_probe where all the other netdev ops are plumbed through, or
- above virtnet_disable_queue_pair which I assume a future queue API implementor would need to call for ndo_queue_stop
I get the feeling you have a much better suggestion in mind though :)
[1]: https://lore.kernel.org/netdev/CACGkMEvWuRjBbc3PvOUpDFkjcby5QNLw5hA_FpNSPyWjkEXD_Q@mail.gmail.com/
Jakub Kicinski
maintainerRe: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 5 lines · On Wed, 5 Mar 2025 17:42:35 -0800 Joe Damato wrote:
Two spots that come to mind are:
- invirtnet_probewhere all the other netdev ops are plumbed through, or
- abovevirtnet_disable_queue_pairwhich I assume a future queue API implementor would need to call forndo_queue_stop
I'd put it next to some call which will have to be inspected. Normally we change napi_disable() to napi_disable_locked() for drivers using the instance lock, so maybe on the napi_disable() line in the refill?
Re: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
Quoted 11 lines · Jakub Kicinski
Quoted 5 lines · On Wed, 5 Mar 2025 17:42:35 -0800 Joe Damato wrote:
Two spots that come to mind are:
- invirtnet_probewhere all the other netdev ops are plumbed through, or
- abovevirtnet_disable_queue_pairwhich I assume a future queue API implementor would need to call forndo_queue_stopI'd put it next to some call which will have to be inspected. Normally we change
napi_disable()tonapi_disable_locked()for drivers using the instance lock, so maybe on thenapi_disable()line in the refill?
Sure, that seems reasonable to me.
Does this comment seem reasonable? I tried to distill what you said in your previous message (thanks for the guidance, btw):
drivers/net/virtio_net.c +12 -0
@@ -2883,6 +2883,18 @@ static void refill_work(struct work_struct *work)for (i =0 ; i < vi->curr_queue_pairs; i++) {struct receive_queue *rq = &vi->rq[i]; +/* +* When queue API support is added in the future and the call +* below becomes napi_disable_locked, this driver will need to +* be refactored. +* +* One possible solution would be to: +* - cancel refill_work with cancel_delayed_work (note: non-sync) +* - cancel refill_work with cancel_delayed_work_sync in +* virtnet_remove after the netdev is unregistered +* - wrap all of the work in a lock (perhaps vi->refill_lock?) +* - check netif_running() and return early to avoid a race +*/
Jakub Kicinski
maintainerRe: [PATCH net-next v5 3/4] virtio-net: Map NAPIs to queues
drivers/net/virtio_net.c static void refill_work(struct work_struct *work) · line 2897
Quoted 3 lines · On Thu, 6 Mar 2025 09:00:02 -0800 Joe Damato wrote:
drivers/net/virtio_net.c +3
-0+* - wrap all of the work in a lock (perhaps vi->refill_lock?) +* - check netif_running() and return early to avoid a race +*/
probably netdev instance lock is better here, as it will also protect the return value of netif_running(). IOW we need to base the "is the device up" test on some state that's protected by the lock we take.
PATCH v5 4/4 virtio_net: Use persistent NAPI config
In reply to Joe DamatoUse persistent NAPI config so that NAPI IDs are not renumbered as queue counts change.
$ sudo ethtool -l ens4 | tail -5 | egrep -i '(current|combined)' Current hardware settings: Combined: 4
$ ./tools/net/ynl/pyynl/cli.py \
--spec Documentation/netlink/specs/netdev.yaml \
--dump queue-get --json='{"ifindex": 2}'
[{'id': 0, 'ifindex': 2, 'napi-id': 8193, 'type': 'rx'},
{'id': 1, 'ifindex': 2, 'napi-id': 8194, 'type': 'rx'},
{'id': 2, 'ifindex': 2, 'napi-id': 8195, 'type': 'rx'},
{'id': 3, 'ifindex': 2, 'napi-id': 8196, 'type': 'rx'},
{'id': 0, 'ifindex': 2, 'type': 'tx'},
{'id': 1, 'ifindex': 2, 'type': 'tx'},
{'id': 2, 'ifindex': 2, 'type': 'tx'},
{'id': 3, 'ifindex': 2, 'type': 'tx'}]Now adjust the queue count, note that the NAPI IDs are not renumbered:
$ sudo ethtool -L ens4 combined 1
$ ./tools/net/ynl/pyynl/cli.py \
--spec Documentation/netlink/specs/netdev.yaml \
--dump queue-get --json='{"ifindex": 2}'
[{'id': 0, 'ifindex': 2, 'napi-id': 8193, 'type': 'rx'},
{'id': 0, 'ifindex': 2, 'type': 'tx'}]$ sudo ethtool -L ens4 combined 8
$ ./tools/net/ynl/pyynl/cli.py \
--spec Documentation/netlink/specs/netdev.yaml \
--dump queue-get --json='{"ifindex": 2}'
[{'id': 0, 'ifindex': 2, 'napi-id': 8193, 'type': 'rx'},
{'id': 1, 'ifindex': 2, 'napi-id': 8194, 'type': 'rx'},
{'id': 2, 'ifindex': 2, 'napi-id': 8195, 'type': 'rx'},
{'id': 3, 'ifindex': 2, 'napi-id': 8196, 'type': 'rx'},
{'id': 4, 'ifindex': 2, 'napi-id': 8197, 'type': 'rx'},
{'id': 5, 'ifindex': 2, 'napi-id': 8198, 'type': 'rx'},
{'id': 6, 'ifindex': 2, 'napi-id': 8199, 'type': 'rx'},
{'id': 7, 'ifindex': 2, 'napi-id': 8200, 'type': 'rx'},
[...]--- drivers/net/virtio_net.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/net/virtio_net.c +3 -2
@@ -6447,8 +6447,9 @@ static int virtnet_alloc_queues(struct virtnet_info *vi)INIT_DELAYED_WORK (&vi->refill, refill_work);for (i =0 ; i < vi->max_queue_pairs; i++) { vi->rq[i].pages =NULL ; -netif_napi_add_weight (vi->dev, &vi->rq[i].napi, virtnet_poll, - napi_weight); +netif_napi_add_config (vi->dev, &vi->rq[i].napi, virtnet_poll, + i); + vi->rq[i].napi.weight = napi_weight;netif_napi_add_tx_weight (vi->dev, &vi->sq[i].napi, virtnet_poll_tx, napi_tx ? napi_weight :0 );
Signature · 1 line
2.45.2
I also tested this series of patches v5 with virtio-net regression tests, everything works fine.
Quoted 97 lines · Joe Damato
Greetings:
Welcome to v5. Patches 1, 2, and 4 have no functional changes only updated tags. Patch 3 was refactored as requested by Jason. See the changelog below and the commit message for details.
Jakub recently commented [1] that I should not hold this series on virtio-net linking queues to NAPIs behind other important work that is on-going and suggested I re-spin, so here we are :)
As per the discussion on the v3 [2], now both RX and TX NAPIs use the API to link queues to NAPIs. Since TX-only NAPIs don't have a NAPI ID, commit
6597e8d35851("netdev-genl: Elidenapi_idwhen not present") now correctly elides the TX-only NAPIs (instead of printing zero) when the queues and NAPIs are linked.As per the discussion on the v4 [3], patch 3 has been refactored to hold RTNL only in the specific locations which need it as Jason requested.
See the commit message of patch 3 for an example of how to get the NAPI to queue mapping information.
See the commit message of patch 4 for an example of how NAPI IDs are persistent despite queue count changes.
Thanks,
Joe[1]: https://lore.kernel.org/netdev/[email protected]/
[2]: https://lore.kernel.org/netdev/[email protected]/
[3]: https://lore.kernel.org/netdev/CACGkMEv=ejJnOWDnAu7eULLvrqXjkMkTL4cbi-uCTUhCpKN_GA@mail.gmail.com/v5:
- Patch 1 added Acked-by's from Michael and Jason. Added Tested-by from Lei. No functional changes.
- Patch 2 added Acked-by's from Michael and Jason. Added Tested-by from Lei. No functional changes.
- Patch 3:
- Refactored as Jason requested, eliminating thevirtnet_queue_set_napihelper entirely, and explicitly holding RTNL in the 3 locations where needed (refill_work, freeze, and restore).
- Commit message updated to outline the known paths at the time the commit was written.
- Patch 4 added Acked-by from Michael. Added Tested-by from Lei. No functional changes.v4: https://lore.kernel.org/lkml/[email protected]/
- Dropped Jakub's patch (previously patch 1).
- Significant refactor from v3 affecting patches 1-3.
- Patch 4 added tags from Jason and Gerhard.rfcv3: https://lore.kernel.org/netdev/[email protected]/
- patch 3:
- Removed the xdp checks completely, as Gerhard Engleder pointed out, they are likely not necessary.- patch 4:
- Added Xuan Zhuo's Reviewed-by.v2: https://lore.kernel.org/netdev/[email protected]/
- patch 1:
- New in the v2 from Jakub.- patch 2:
- Previously patch 1, unchanged from v1.
- Added Gerhard Engleder's Reviewed-by.
- Added Lei Yang's Tested-by.- patch 3:
- Introducedvirtnet_napi_disableto eliminate duplicated code invirtnet_xdp_set,virtnet_rx_pause,virtnet_disable_queue_pair,refill_workas suggested by Jason Wang.
- As a result of the above refactor, dropped Reviewed-by and Tested-by from patch 3.- patch 4:
- New in v2. Adds persistent NAPI configuration. See commit message for more details.v1: https://lore.kernel.org/netdev/[email protected]/
Joe Damato (4):
virtio-net: Refactornapi_enablepaths
virtio-net: Refactornapi_disablepaths
virtio-net: Map NAPIs to queues
virtio_net: Use persistent NAPI configdrivers/net/virtio_net.c | 95 ++++++++++++++++++++++++++++------------ 1 file changed, 67 insertions(+), 28 deletions(-)base-commit:
7fe0353606d77a32c4c7f2814833dd1c043ebdd2--
2.45.2