virtio-net: support dynamic coalescing moderation
Subsystems (4)
v8 cover virtio-net: support dynamic coalescing moderation
Now, virtio-net already supports per-queue moderation parameter setting. Based on this, we use the linux dimlib to support dynamic coalescing moderation for virtio-net.
Due to some scheduling issues, we only support and test the rx dim.
Some test results:
I. Sockperf UDP
=================================================
1. Envrxq_0 with affinity to cpu_0.
2. Cmd
client: taskset -c 0 sockperf tp -p 8989 -i $IP -t 10 -m 16B
server: taskset -c 0 sockperf sr -p 8989
3. Result
dim off: 1143277.00 rxpps, throughput 17.844 MBps, cpu is 100%. dim on: 1124161.00 rxpps, throughput 17.610 MBps, cpu is 83.5%.
=================================================
II. Redis
=================================================
1. Env
There are 8 rxqs, and rxq_i with affinity to cpu_i.
2. Result
When all cpus are 100%, ops/sec of memtier_benchmark client is
dim off: 978437.23 dim on: 1143638.28
=================================================
III. Nginx
=================================================
1. Env
There are 8 rxqs and rxq_i with affinity to cpu_i.
2. Result
When all cpus are 100%, requests/sec of wrk client is
dim off: 877931.67 dim on: 1019160.31
=================================================
IV. Latency of sockperf udp
=================================================
1. Rx cmd
taskset -c 0 sockperf sr -p 8989
2. Tx cmd
taskset -c 0 sockperf pp -i ${ip} -p 8989 -t 10
After running this cmd 5 times and averaging the results,
3. Result
dim off: 17.7735 usec dim on: 18.0110 usec
=================================================
Changelog:v7->v8:
- Add select DIMLIB.
v6->v7:
- Drop the patch titled "spin lock for ctrl cmd access"
- Use rtnl_trylock to avoid the deadlock.
v5->v6:
- Add patch(4/5): spin lock for ctrl cmd access
- Patch(5/5):
- Use spin lock and cancel_work_sync to synchronize
v4->v5:
- Patch(4/4):
- Fix possible synchronization issues with cancel_work_sync.
- Reduce if/else nesting levels
v3->v4:
- Patch(5/5): drop.
v2->v3:
- Patch(4/5): some minor modifications.
v1->v2:
- Patch(2/5): a minor fix.
- Patch(4/5):
- improve the judgment of dim switch conditions.
- Cancel the work when vq reset.
- Patch(5/5): drop the tx dim implementation.
Heng Qi (4):
virtio-net: returns whether napi is complete
virtio-net: separate rx/tx coalescing moderation cmds
virtio-net: extract virtqueue coalescig cmd for reuse
virtio-net: support rx netdim
drivers/net/Kconfig | 1 + drivers/net/virtio_net.c | 297 ++++++++++++++++++++++++++++++++------- 2 files changed, 249 insertions(+), 49 deletions(-)
Signature · 1 line
2.19.1.6.gb485710b
PATCH v8 1/4 virtio-net: returns whether napi is complete
rx netdim needs to count the traffic during a complete napi process, and start updating and comparing samples to make decisions after the napi ends. Let virtqueue_napi_complete() return true if napi is done, otherwise vice versa.
--- drivers/net/virtio_net.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/net/virtio_net.c +5 -1
@@ -431,7 +431,7 @@ static void virtqueue_napi_schedule(struct napi_struct *napi, } } -static void virtqueue_napi_complete (struct napi_struct *napi, +static bool virtqueue_napi_complete (struct napi_struct *napi,struct virtqueue *vq,int processed) {int opaque;
@@ -440,9 +440,13 @@ static void virtqueue_napi_complete(struct napi_struct *napi,if (napi_complete_done (napi, processed)) {if (unlikely (virtqueue_poll (vq, opaque)))virtqueue_napi_schedule (napi, vq); +else +return true; }else {virtqueue_disable_cb (vq); } + +return false; }static void skb_xmit_done (struct virtqueue *vq)
Signature · 1 line
2.19.1.6.gb485710b
PATCH v8 2/4 virtio-net: separate rx/tx coalescing moderation cmds
In reply to Heng QiThis patch separates the rx and tx global coalescing moderation commands to support netdim switches in subsequent patches.
--- drivers/net/virtio_net.c | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)
drivers/net/virtio_net.c +28 -3
@@ -3266,10 +3266,10 @@ static int virtnet_get_link_ksettings(struct net_device *dev,return 0 ; } -static int virtnet_send_notf_coal_cmds (struct virtnet_info *vi, -struct ethtool_coalesce *ec) +static int virtnet_send_tx_notf_coal_cmds (struct virtnet_info *vi, +struct ethtool_coalesce *ec) { -struct scatterlist sgs_tx, sgs_rx; +struct scatterlist sgs_tx;int i; vi->ctrl->coal_tx.tx_usecs =cpu_to_le32 (ec->tx_coalesce_usecs);
@@ -3289,6 +3289,15 @@ static int virtnet_send_notf_coal_cmds(struct virtnet_info *vi, vi->sq[i].intr_coal.max_packets = ec->tx_max_coalesced_frames; } +return 0 ; +} + +static int virtnet_send_rx_notf_coal_cmds (struct virtnet_info *vi, +struct ethtool_coalesce *ec) +{ +struct scatterlist sgs_rx; +int i; + vi->ctrl->coal_rx.rx_usecs =cpu_to_le32 (ec->rx_coalesce_usecs); vi->ctrl->coal_rx.rx_max_packets =cpu_to_le32 (ec->rx_max_coalesced_frames);sg_init_one (&sgs_rx, &vi->ctrl->coal_rx,sizeof (vi->ctrl->coal_rx));
@@ -3309,6 +3318,22 @@ static int virtnet_send_notf_coal_cmds(struct virtnet_info *vi,return 0 ; } +static int virtnet_send_notf_coal_cmds (struct virtnet_info *vi, +struct ethtool_coalesce *ec) +{ +int err; + + err =virtnet_send_tx_notf_coal_cmds (vi, ec); +if (err) +return err; + + err =virtnet_send_rx_notf_coal_cmds (vi, ec); +if (err) +return err; + +return 0 ; +} +static int virtnet_send_ctrl_coal_vq_cmd (struct virtnet_info *vi,u16 vqn,u32 max_usecs,u32 max_packets) {
Signature · 1 line
2.19.1.6.gb485710b
PATCH v8 3/4 virtio-net: extract virtqueue coalescig cmd for reuse
In reply to Heng QiExtract commands to set virtqueue coalescing parameters for reuse by ethtool -Q, vq resize and netdim.
--- drivers/net/virtio_net.c | 106 +++++++++++++++++++++++---------------- 1 file changed, 64 insertions(+), 42 deletions(-)
drivers/net/virtio_net.c +64 -42
@@ -2849,6 +2849,58 @@ static void virtnet_cpu_notif_remove(struct virtnet_info *vi) &vi->node_dead); } +static int virtnet_send_ctrl_coal_vq_cmd (struct virtnet_info *vi, +u16 vqn,u32 max_usecs,u32 max_packets) +{ +struct scatterlist sgs; + + vi->ctrl->coal_vq.vqn =cpu_to_le16 (vqn); + vi->ctrl->coal_vq.coal.max_usecs =cpu_to_le32 (max_usecs); + vi->ctrl->coal_vq.coal.max_packets =cpu_to_le32 (max_packets); +sg_init_one (&sgs, &vi->ctrl->coal_vq,sizeof (vi->ctrl->coal_vq)); + +if (!virtnet_send_command (vi,VIRTIO_NET_CTRL_NOTF_COAL , +VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET , + &sgs)) +return -EINVAL ; + +return 0 ; +} + +static int virtnet_send_rx_ctrl_coal_vq_cmd (struct virtnet_info *vi, +u16 queue,u32 max_usecs, +u32 max_packets) +{ +int err; + + err =virtnet_send_ctrl_coal_vq_cmd (vi,rxq2vq (queue), + max_usecs, max_packets); +if (err) +return err; + + vi->rq[queue].intr_coal.max_usecs = max_usecs; + vi->rq[queue].intr_coal.max_packets = max_packets; + +return 0 ; +} + +static int virtnet_send_tx_ctrl_coal_vq_cmd (struct virtnet_info *vi, +u16 queue,u32 max_usecs, +u32 max_packets) +{ +int err; + + err =virtnet_send_ctrl_coal_vq_cmd (vi,txq2vq (queue), + max_usecs, max_packets); +if (err) +return err; + + vi->sq[queue].intr_coal.max_usecs = max_usecs; + vi->sq[queue].intr_coal.max_packets = max_packets; + +return 0 ; +} + static void virtnet_get_ringparam(struct net_device *dev, struct ethtool_ringparam *ring, struct kernel_ethtool_ringparam *kernel_ring,
@@ -2906,14 +2958,11 @@ static int virtnet_set_ringparam(struct net_device *dev,* through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver * did not set any TX coalescing parameters, to 0. */ - err =virtnet_send_ctrl_coal_vq_cmd (vi,txq2vq (i), - vi->intr_coal_tx.max_usecs, - vi->intr_coal_tx.max_packets); + err =virtnet_send_tx_ctrl_coal_vq_cmd (vi, i, + vi->intr_coal_tx.max_usecs, + vi->intr_coal_tx.max_packets);if (err)return err; - - vi->sq[i].intr_coal.max_usecs = vi->intr_coal_tx.max_usecs; - vi->sq[i].intr_coal.max_packets = vi->intr_coal_tx.max_packets; }if (ring->rx_pending != rx_pending) {
@@ -2922,14 +2971,11 @@ static int virtnet_set_ringparam(struct net_device *dev,return err;/* The reason is same as the transmit virtqueue reset */ - err =virtnet_send_ctrl_coal_vq_cmd (vi,rxq2vq (i), - vi->intr_coal_rx.max_usecs, - vi->intr_coal_rx.max_packets); + err =virtnet_send_rx_ctrl_coal_vq_cmd (vi, i, + vi->intr_coal_rx.max_usecs, + vi->intr_coal_rx.max_packets);if (err)return err; - - vi->rq[i].intr_coal.max_usecs = vi->intr_coal_rx.max_usecs; - vi->rq[i].intr_coal.max_packets = vi->intr_coal_rx.max_packets; } }
@@ -3334,48 +3380,24 @@ static int virtnet_send_notf_coal_cmds(struct virtnet_info *vi,return 0 ; } -static int virtnet_send_ctrl_coal_vq_cmd (struct virtnet_info *vi, -u16 vqn,u32 max_usecs,u32 max_packets) -{ -struct scatterlist sgs; - - vi->ctrl->coal_vq.vqn =cpu_to_le16 (vqn); - vi->ctrl->coal_vq.coal.max_usecs =cpu_to_le32 (max_usecs); - vi->ctrl->coal_vq.coal.max_packets =cpu_to_le32 (max_packets); -sg_init_one (&sgs, &vi->ctrl->coal_vq,sizeof (vi->ctrl->coal_vq)); - -if (!virtnet_send_command (vi,VIRTIO_NET_CTRL_NOTF_COAL , -VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET , - &sgs)) -return -EINVAL ; - -return 0 ; -} -static int virtnet_send_notf_coal_vq_cmds (struct virtnet_info *vi,struct ethtool_coalesce *ec,u16 queue) {int err; - err =virtnet_send_ctrl_coal_vq_cmd (vi,rxq2vq (queue), - ec->rx_coalesce_usecs, - ec->rx_max_coalesced_frames); + err =virtnet_send_rx_ctrl_coal_vq_cmd (vi, queue, + ec->rx_coalesce_usecs, + ec->rx_max_coalesced_frames);if (err)return err; - vi->rq[queue].intr_coal.max_usecs = ec->rx_coalesce_usecs; - vi->rq[queue].intr_coal.max_packets = ec->rx_max_coalesced_frames; - - err =virtnet_send_ctrl_coal_vq_cmd (vi,txq2vq (queue), - ec->tx_coalesce_usecs, - ec->tx_max_coalesced_frames); + err =virtnet_send_tx_ctrl_coal_vq_cmd (vi, queue, + ec->tx_coalesce_usecs, + ec->tx_max_coalesced_frames);if (err)return err; - vi->sq[queue].intr_coal.max_usecs = ec->tx_coalesce_usecs; - vi->sq[queue].intr_coal.max_packets = ec->tx_max_coalesced_frames; -return 0 ; }
Signature · 1 line
2.19.1.6.gb485710b
PATCH v8 4/4 virtio-net: support rx netdim
In reply to Heng QiBy comparing the traffic information in the complete napi processes, let the virtio-net driver automatically adjust the coalescing moderation parameters of each receive queue.
---v7->v8:
- Add select DIMLIB
v6->v7:
- Drop the patch titled "spin lock for ctrl cmd access"
- Use rtnl_trylock to avoid the deadlock.
v5->v6:
- Use spin lock and cancel_work_sync to synchronize
v4->v5:
- Fix possible synchronization issues using cancel_work().
- Reduce if/else nesting levels
v2->v3:
- Some minor modifications.
v1->v2:
- Improved the judgment of dim switch conditions.
- Cancel the work when vq reset.
drivers/net/Kconfig | 1 + drivers/net/virtio_net.c | 176 +++++++++++++++++++++++++++++++++++---- 2 files changed, 163 insertions(+), 14 deletions(-)
drivers/net/Kconfig +1 -0
@@ -434,6 +434,7 @@ config VIRTIO_NET tristate "Virtio network driver" depends on VIRTIO select NET_FAILOVER + select DIMLIB help This is the virtual network driver for virtio. It can be used with QEMU based VMMs (like KVM or Xen). Say Y or M.
drivers/net/virtio_net.c +162 -14
314 lines not on this page · raw message
Signature · 1 line
2.19.1.6.gb485710b
This series was applied to netdev/net-next.git (main) by David S. Miller <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by David S. Miller <[email protected]>:
Quoted 9 lines · On Mon, 11 Dec 2023 18:36:03 +0800 you wrote:
Now, virtio-net already supports per-queue moderation parameter setting. Based on this, we use the linux dimlib to support dynamic coalescing moderation for virtio-net.
Due to some scheduling issues, we only support and test the rx dim.
Some test results:
[...]
Here is the summary with links:
- [net-next,v8,1/4] virtio-net: returns whether napi is complete
https://git.kernel.org/netdev/net-next/c/7949c06ad9a8 - [net-next,v8,2/4] virtio-net: separate rx/tx coalescing moderation cmds https://git.kernel.org/netdev/net-next/c/d7180080ddf7 - [net-next,v8,3/4] virtio-net: extract virtqueue coalescig cmd for reuse https://git.kernel.org/netdev/net-next/c/1db43c0818e2 - [net-next,v8,4/4] virtio-net: support rx netdim https://git.kernel.org/netdev/net-next/c/6208799553a8
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html