Skip to contentSkip to conversation

net/vhost: support asynchronous data path

RFC In review Yuan Wang 2 participants 5 messages Dec 16, 2022 – Jan 6, 2023

Subsystems (2)
lore
Share & tools
lore

Yuan Wang

author
#0 lore raw

RFC 1/1 net/vhost: support asynchronous data path

Vhost asynchronous data-path offloads packet copy from the CPU to the DMA engine. As a result, large packet copy can be accelerated by the DMA engine, and vhost can free CPU cycles for higher level functions.

In this patch, we enable asynchronous data-path for vhostpmd. Asynchronous data path is enabled per tx/rx queue, and users need to specify the DMA device used by the tx/rx queue. Each tx/rx queue only supports to use one DMA device, but one DMA device can be shared among multiple tx/rx queues of different vhost PMD ports.

Two PMD parameters are added:
- dmas: specify the used DMA device for a tx/rx queue.
(Default: no queues enable asynchronous data path)
- dma-ring-size: DMA ring size.

	(Default: 4096).

Here is an example:
--vdev 'eth_vhost0,iface=./s0,dmas=[txq0@0000:00.01.0;rxq0@0000:00.01.1],dma-ring-size=4096'

---
 drivers/net/vhost/meson.build     |   1 +
 drivers/net/vhost/rte_eth_vhost.c | 512 ++++++++++++++++++++++++++++--
 drivers/net/vhost/rte_eth_vhost.h |  15 +
 drivers/net/vhost/version.map     |   7 +
 drivers/net/vhost/vhost_testpmd.c |  67 ++++
 5 files changed, 569 insertions(+), 33 deletions(-)
 create mode 100644 drivers/net/vhost/vhost_testpmd.c
drivers/net/vhost/meson.build +1 -0
index f481a3a4b8..22a0ab3a58 100644
@@ -9,4 +9,5 @@ endif
 
 deps += 'vhost'
 sources = files('rte_eth_vhost.c')
+testpmd_sources = files('vhost_testpmd.c')
 headers = files('rte_eth_vhost.h')
drivers/net/vhost/rte_eth_vhost.c +479 -33

729 lines not on this page · raw message

drivers/net/vhost/rte_eth_vhost.h +15 -0
index 0e68b9f668..146c98803d 100644
@@ -52,6 +52,21 @@ int rte_eth_vhost_get_queue_event(uint16_t port_id,
  */
 int rte_eth_vhost_get_vid_from_port_id(uint16_t port_id);
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * By default, rte_vhost_poll_enqueue_completed() is called in Rx path.
+ * This function enables Tx path, rather than Rx path, to poll completed
+ * packets for vhost async enqueue operations. Note that virtio may never
+ * receive DMA completed packets if there are no more Tx operations.
+ *
+ * @param enable
+ *  True indicates Tx path to call rte_vhost_poll_enqueue_completed().
+ */
+__rte_experimental
+void rte_eth_vhost_async_tx_poll_completed(bool enable);
+
 #ifdef __cplusplus
 }
 #endif
drivers/net/vhost/version.map +7 -0
index e42c89f1eb..0a40441227 100644
@@ -6,3 +6,10 @@ DPDK_23 {
 
 	local: *;
 };
+
+EXPERIMENTAL {
+	global:
+
+	# added in 22.11
+	rte_eth_vhost_async_tx_poll_completed;
+};
drivers/net/vhost/vhost_testpmd.c new file +67 -0

67 lines not on this page · raw message

Signature · 1 line
2.25.1
#1 lore raw

Hi Yuan,

Quoted 23 lines · Yuan Wang

Vhost asynchronous data-path offloads packet copy from the CPU to the DMA engine. As a result, large packet copy can be accelerated by the DMA engine, and vhost can free CPU cycles for higher level functions.

In this patch, we enable asynchronous data-path for vhostpmd. Asynchronous data path is enabled per tx/rx queue, and users need to specify the DMA device used by the tx/rx queue. Each tx/rx queue only supports to use one DMA device, but one DMA device can be shared among multiple tx/rx queues of different vhost PMD ports.

Two PMD parameters are added:
- dmas: specify the used DMA device for a tx/rx queue.
(Default: no queues enable asynchronous data path)
- dma-ring-size: DMA ring size.

	(Default: 4096).

Here is an example:
--vdev 'eth_vhost0,iface=./s0,dmas=[txq0@0000:00.01.0;rxq0@0000:00.01.1],dma-ring-size=4096'

Quoted 9 lines
---
  drivers/net/vhost/meson.build     |   1 +
  drivers/net/vhost/rte_eth_vhost.c | 512 ++++++++++++++++++++++++++++--
  drivers/net/vhost/rte_eth_vhost.h |  15 +
  drivers/net/vhost/version.map     |   7 +
  drivers/net/vhost/vhost_testpmd.c |  67 ++++
  5 files changed, 569 insertions(+), 33 deletions(-)

create mode 100644 drivers/net/vhost/vhost_testpmd.c

This RFC is identical to the v5 that you sent for last release, and so the comments I made on it are still valid.

Is this intentionally re-sent?

Regards,
Maxime

#2 lore raw

Hi Maxime,

Sorry about not being clear about the intentions.
The patch is for a whitepaper, we use it for tests and we need to attach the patch link. Maybe I should set the patch state to superseded?

Thanks,
Yuan

Quoted 53 lines

-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Monday, January 2, 2023 6:59 PM
To: Wang, YuanX <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>
Subject: Re: [RFC] net/vhost: support asynchronous data path

Hi Yuan,

Quoted 8 lines · Yuan Wang

Vhost asynchronous data-path offloads packet copy from the CPU to the DMA engine. As a result, large packet copy can be accelerated by the DMA engine, and vhost can free CPU cycles for higher level functions.

In this patch, we enable asynchronous data-path for vhostpmd. Asynchronous data path is enabled per tx/rx queue, and users need to specify the DMA device used by the tx/rx queue. Each tx/rx queue only supports to use one DMA device, but one DMA device can be shared

among

Quoted 10 lines

multiple tx/rx queues of different vhost PMD ports.

Two PMD parameters are added:
- dmas: specify the used DMA device for a tx/rx queue.
(Default: no queues enable asynchronous data path)
- dma-ring-size: DMA ring size.

	(Default: 4096).

Here is an example:
--vdev

'eth_vhost0,iface=./s0,dmas=[txq0@0000:00.01.0;rxq0@0000:00.01.1],dma- ring-size=4096'

Quoted 7 lines
---
  drivers/net/vhost/meson.build     |   1 +
  drivers/net/vhost/rte_eth_vhost.c | 512

++++++++++++++++++++++++++++--

Quoted 6 lines
  drivers/net/vhost/rte_eth_vhost.h |  15 +
  drivers/net/vhost/version.map     |   7 +
  drivers/net/vhost/vhost_testpmd.c |  67 ++++
  5 files changed, 569 insertions(+), 33 deletions(-)

create mode 100644 drivers/net/vhost/vhost_testpmd.c

This RFC is identical to the v5 that you sent for last release, and so the comments I made on it are still valid.

Is this intentionally re-sent?

Regards,
Maxime

#3 lore raw

Hi,

Quoted 4 lines · YuanX

Hi Maxime,

Sorry about not being clear about the intentions.
The patch is for a whitepaper, we use it for tests and we need to attach the patch link.

Thanks for the clarification.

Quoted 1 line

Maybe I should set the patch state to superseded?

I think setting it to rejected would be the appropriate state, as there are no patch that supersedes it.

Thanks,
Maxime

Quoted 57 lines

Thanks,
Yuan

Quoted 53 lines

-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Monday, January 2, 2023 6:59 PM
To: Wang, YuanX <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>
Subject: Re: [RFC] net/vhost: support asynchronous data path

Hi Yuan,

Quoted 8 lines · Yuan Wang

Vhost asynchronous data-path offloads packet copy from the CPU to the DMA engine. As a result, large packet copy can be accelerated by the DMA engine, and vhost can free CPU cycles for higher level functions.

In this patch, we enable asynchronous data-path for vhostpmd. Asynchronous data path is enabled per tx/rx queue, and users need to specify the DMA device used by the tx/rx queue. Each tx/rx queue only supports to use one DMA device, but one DMA device can be shared

among

Quoted 10 lines

multiple tx/rx queues of different vhost PMD ports.

Two PMD parameters are added:
- dmas: specify the used DMA device for a tx/rx queue.
(Default: no queues enable asynchronous data path)
- dma-ring-size: DMA ring size.

	(Default: 4096).

Here is an example:
--vdev

'eth_vhost0,iface=./s0,dmas=[txq0@0000:00.01.0;rxq0@0000:00.01.1],dma- ring-size=4096'

Quoted 7 lines
---
   drivers/net/vhost/meson.build     |   1 +
   drivers/net/vhost/rte_eth_vhost.c | 512

++++++++++++++++++++++++++++--

Quoted 6 lines
   drivers/net/vhost/rte_eth_vhost.h |  15 +
   drivers/net/vhost/version.map     |   7 +
   drivers/net/vhost/vhost_testpmd.c |  67 ++++
   5 files changed, 569 insertions(+), 33 deletions(-)

create mode 100644 drivers/net/vhost/vhost_testpmd.c

This RFC is identical to the v5 that you sent for last release, and so the comments I made on it are still valid.

Is this intentionally re-sent?

Regards,
Maxime

#4 lore raw

Hi,

Quoted 14 lines

-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Friday, January 6, 2023 5:34 PM
To: Wang, YuanX <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>
Subject: Re: [RFC] net/vhost: support asynchronous data path

Hi,

Quoted 4 lines · YuanX

Hi Maxime,

Sorry about not being clear about the intentions.
The patch is for a whitepaper, we use it for tests and we need to attach the

Quoted 8 lines

patch link.

Thanks for the clarification.

Quoted 1 line

Maybe I should set the patch state to superseded?

I think setting it to rejected would be the appropriate state, as there are no patch that supersedes it.

Ok, thanks for your advice.

Regards,
Yuan

Quoted 64 lines

Thanks,
Maxime

Quoted 34 lines

Thanks,
Yuan

Quoted 31 lines

-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Monday, January 2, 2023 6:59 PM
To: Wang, YuanX <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>
Subject: Re: [RFC] net/vhost: support asynchronous data path

Hi Yuan,

Quoted 9 lines · Yuan Wang

Vhost asynchronous data-path offloads packet copy from the CPU to the DMA engine. As a result, large packet copy can be accelerated by the DMA engine, and vhost can free CPU cycles for higher level functions.

In this patch, we enable asynchronous data-path for vhostpmd. Asynchronous data path is enabled per tx/rx queue, and users need to specify the DMA device used by the tx/rx queue. Each tx/rx queue only supports to use one DMA device, but one DMA device can be shared

among

Quoted 10 lines

multiple tx/rx queues of different vhost PMD ports.

Two PMD parameters are added:
- dmas: specify the used DMA device for a tx/rx queue.
(Default: no queues enable asynchronous data path)
- dma-ring-size: DMA ring size.

	(Default: 4096).

Here is an example:
--vdev

'eth_vhost0,iface=./s0,dmas=[txq0@0000:00.01.0;rxq0@0000:00.01.1],dma

Quoted 25 lines
Quoted 24 lines

-
ring-size=4096'

Quoted 7 lines
---
   drivers/net/vhost/meson.build     |   1 +
   drivers/net/vhost/rte_eth_vhost.c | 512

++++++++++++++++++++++++++++--

Quoted 6 lines
   drivers/net/vhost/rte_eth_vhost.h |  15 +
   drivers/net/vhost/version.map     |   7 +
   drivers/net/vhost/vhost_testpmd.c |  67 ++++
   5 files changed, 569 insertions(+), 33 deletions(-)

create mode 100644 drivers/net/vhost/vhost_testpmd.c

This RFC is identical to the v5 that you sent for last release, and so the comments I made on it are still valid.

Is this intentionally re-sent?

Regards,
Maxime