Skip to contentSkip to conversation

doc: update async enqueue API usage

PATCH Superseded [email protected] 4 participants 9 messages Jun 17, 2022 – Jul 1, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 doc: update async enqueue API usage

From: Xuan Ding <[email protected]>

This patch updates the correct usage for async enqueue APIs. The rte_vhost_poll_enqueue_completed() needs to be called in time to avoid packet loss due to full dma ring.

---
 doc/guides/prog_guide/vhost_lib.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +9 -0
index cd3f6caa9a..9cdd7da154 100644
@@ -488,3 +488,12 @@ For PA mode, page by page mapping may exceed IOMMU's max capability,
 better to use 1G guest hugepage.
 
 For UIO driver, any VFIO related error message can be ignored.
+
+Async enqueue API usage
+-----------------------
+
+In async enqueue path, rte_vhost_poll_enqueue_completed() needs to be
+called in time. Otherwise, calling rte_vhost_submit_enqueue_burst()
+all the time will cause the DMA ring to be full, which will result
+in packet loss eventually.
+
Signature · 1 line
2.17.1
#1 lore raw

Hi Xuan,

Quoted 28 lines

-----Original Message-----
From: Ding, Xuan <[email protected]>
Sent: Friday, June 17, 2022 5:34 PM
To: [email protected]; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>
Subject: [PATCH] doc: update async enqueue API usage

From: Xuan Ding <[email protected]>

This patch updates the correct usage for async enqueue APIs. The rte_vhost_poll_enqueue_completed() needs to be called in time to avoid packet loss due to full dma ring.

---
 doc/guides/prog_guide/vhost_lib.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +0 -0
index cd3f6caa9a..9cdd7da154 100644
@@ -488,3 +488,12 @@ For PA mode, page by page mapping may exceed

IOMMU's max capability, better to use 1G guest hugepage.

For UIO driver, any VFIO related error message can be ignored.

doc/guides/prog_guide/vhost_lib.rst For PA mode, page by page mapping may exceed IOMMU's max capability, · line 493

Quoted 3 lines
doc/guides/prog_guide/vhost_lib.rst +3 -0
+
+Async enqueue API usage
+-----------------------

I think it's better to make it as a sub-section of "Vhost asynchronous data path".

doc/guides/prog_guide/vhost_lib.rst

Quoted 5 lines
doc/guides/prog_guide/vhost_lib.rst +5 -0
+
+In async enqueue path, rte_vhost_poll_enqueue_completed() needs to be
+called in time. Otherwise, calling rte_vhost_submit_enqueue_burst() all
+the time will cause the DMA ring to be full, which will result in
+packet loss eventually.

There are two main reasons of calling rte_vhost_poll_enqueue_completed() timely, IMO:
1. notify the guest of DMA copy completed packets;
2. in case of DMA ring size is smaller than vring size, it can avoid packet drop caused by DMA ring full.

Thanks,
Jiayu

Quoted 3 lines

+
--
2.17.1

author
#2 lore raw

Hi Jiayu,

Quoted 37 lines

-----Original Message-----
From: Hu, Jiayu <[email protected]>
Sent: Monday, June 20, 2022 9:43 AM
To: Ding, Xuan <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>
Cc: [email protected]
Subject: RE: [PATCH] doc: update async enqueue API usage

Hi Xuan,

Quoted 27 lines

-----Original Message-----
From: Ding, Xuan <[email protected]>
Sent: Friday, June 17, 2022 5:34 PM
To: [email protected]; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>
Subject: [PATCH] doc: update async enqueue API usage

From: Xuan Ding <[email protected]>

This patch updates the correct usage for async enqueue APIs. The rte_vhost_poll_enqueue_completed() needs to be called in time to avoid packet loss due to full dma ring.

---
 doc/guides/prog_guide/vhost_lib.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +0 -0
index cd3f6caa9a..9cdd7da154 100644
@@ -488,3 +488,12 @@ For PA mode, page by page mapping may exceed

IOMMU's max capability, better to use 1G guest hugepage.

Quoted 7 lines

doc/guides/prog_guide/vhost_lib.rst For PA mode, page by page mapping may exceed IOMMU's max capability, · line 493

Quoted 4 lines

For UIO driver, any VFIO related error message can be ignored.

doc/guides/prog_guide/vhost_lib.rst +3 -0
+
+Async enqueue API usage
+-----------------------

I think it's better to make it as a sub-section of "Vhost asynchronous data path".

That's a good idea, I will add this section as a sub-section accordingly.

Quoted 12 lines

doc/guides/prog_guide/vhost_lib.rst For PA mode, page by page mapping may exceed IOMMU's max capability,

Quoted 6 lines
doc/guides/prog_guide/vhost_lib.rst +5 -0
+
+In async enqueue path, rte_vhost_poll_enqueue_completed() needs to be
+called in time. Otherwise, calling rte_vhost_submit_enqueue_burst()
+all the time will cause the DMA ring to be full, which will result in
+packet loss eventually.

There are two main reasons of calling rte_vhost_poll_enqueue_completed() timely, IMO:
1. notify the guest of DMA copy completed packets; 2. in case of DMA ring size is smaller than vring size, it can avoid packet drop caused by DMA ring full.

Please see doc enriched in next version.

Thanks,
Xuan

Quoted 9 lines

Thanks,
Jiayu

Quoted 3 lines

+
--
2.17.1

#3 lore raw
In reply to Hu, Jiayu

Hi Xuan,

Quoted 45 lines

-----Original Message-----
From: Hu, Jiayu
Sent: Monday, June 20, 2022 9:43 AM
To: Ding, Xuan <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>
Cc: [email protected]
Subject: RE: [PATCH] doc: update async enqueue API usage

Hi Xuan,

Quoted 28 lines

-----Original Message-----
From: Ding, Xuan <[email protected]>
Sent: Friday, June 17, 2022 5:34 PM
To: [email protected]; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>
Subject: [PATCH] doc: update async enqueue API usage

From: Xuan Ding <[email protected]>

This patch updates the correct usage for async enqueue APIs. The rte_vhost_poll_enqueue_completed() needs to be called in time to avoid packet loss due to full dma ring.

---
 doc/guides/prog_guide/vhost_lib.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +0 -0
index cd3f6caa9a..9cdd7da154 100644
@@ -488,3 +488,12 @@ For PA mode, page by page mapping may exceed

IOMMU's max capability, better to use 1G guest hugepage.

For UIO driver, any VFIO related error message can be ignored.

doc/guides/prog_guide/vhost_lib.rst For PA mode, page by page mapping may exceed IOMMU's max capability, · line 493

Quoted 3 lines
doc/guides/prog_guide/vhost_lib.rst +3 -0
+
+Async enqueue API usage
+-----------------------

I think it's better to make it as a sub-section of "Vhost asynchronous data path".

Quoted 11 lines

doc/guides/prog_guide/vhost_lib.rst For PA mode, page by page mapping may exceed IOMMU's max capability,

Quoted 5 lines
doc/guides/prog_guide/vhost_lib.rst +5 -0
+
+In async enqueue path, rte_vhost_poll_enqueue_completed() needs to be
+called in time. Otherwise, calling rte_vhost_submit_enqueue_burst()
+all the time will cause the DMA ring to be full, which will result in
+packet loss eventually.

There are two main reasons of calling rte_vhost_poll_enqueue_completed() timely, IMO:
1. notify the guest of DMA copy completed packets; 2. in case of DMA ring size is smaller than vring size, it can avoid packet drop caused by DMA ring full.

I'd like to clarify #2 a little: not only for the case that DMA ring size < vring size. Even if DMA ring size > vring size, it's possible dropping packets as a result of DMA ring full.

Thanks,
Jiayu

Quoted 8 lines

Thanks,
Jiayu

Quoted 3 lines

+
--
2.17.1

#4 lore raw

PATCH v2 1/1 doc: update async enqueue API usage

In reply to [email protected]

From: Xuan Ding <[email protected]>

This patch updates the correct usage for async enqueue APIs. The rte_vhost_poll_enqueue_completed() needs to be called in time to notify the guest of completed packets and avoid packet loss.

---
v2:
* refine doc and commit log

---
 doc/guides/prog_guide/vhost_lib.rst | 8 ++++++++
 1 file changed, 8 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +8 -0
index cd3f6caa9a..628db3fc38 100644
@@ -480,6 +480,14 @@ the same vring with their own DMA virtual channels. Besides, the number
 of DMA devices is limited. For the purpose of scaling, it's necessary to
 support sharing DMA channels among vrings.
 
+* Async enqueue API usage
+
+  In async enqueue path, rte_vhost_poll_enqueue_completed() needs to be
+  called in time to notify the guest of DMA copy completed packets.
+  Moreover, calling rte_vhost_submit_enqueue_burst() all the time but
+  not poll completed will cause the DMA ring to be full, which will
+  result in packet loss eventually.
+
 Recommended IOVA mode in async datapath
 ---------------------------------------
 
Signature · 1 line
2.17.1
#5 lore raw

Hi Xuan,

Quoted 33 lines

-----Original Message-----
From: Ding, Xuan <[email protected]>
Sent: Tuesday, June 21, 2022 3:22 PM
To: [email protected]; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>
Subject: [PATCH v2] doc: update async enqueue API usage

From: Xuan Ding <[email protected]>

This patch updates the correct usage for async enqueue APIs. The rte_vhost_poll_enqueue_completed() needs to be called in time to notify the guest of completed packets and avoid packet loss.

---
v2:
* refine doc and commit log

---
 doc/guides/prog_guide/vhost_lib.rst | 8 ++++++++
 1 file changed, 8 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +0 -0
index cd3f6caa9a..628db3fc38 100644
@@ -480,6 +480,14 @@ the same vring with their own DMA virtual channels.

Besides, the number
of DMA devices is limited. For the purpose of scaling, it's necessary to support sharing DMA channels among vrings.

doc/guides/prog_guide/vhost_lib.rst the same vring with their own DMA virtual channels. Besides, the number · line 490

Quoted 8 lines
doc/guides/prog_guide/vhost_lib.rst +8 -0
+* Async enqueue API usage
+
+  In async enqueue path, rte_vhost_poll_enqueue_completed() needs to be
+  called in time to notify the guest of DMA copy completed packets.
+  Moreover, calling rte_vhost_submit_enqueue_burst() all the time but
+  not poll completed will cause the DMA ring to be full, which will
+  result in packet loss eventually.
+

This subsection seems cleaner, could you also help to make 'Recommended IOVA mode in async datapath' as a sub-section?

Another patch or not, either works for me.

Thanks,
Chenbo

Quoted 5 lines

Recommended IOVA mode in async datapath
---------------------------------------

--
2.17.1

author
#6 lore raw

Hi Chenbo,

Quoted 54 lines

-----Original Message-----
From: Xia, Chenbo <[email protected]>
Sent: Tuesday, June 21, 2022 9:23 PM
To: Ding, Xuan <[email protected]>; [email protected]
Cc: [email protected]; Hu, Jiayu <[email protected]>
Subject: RE: [PATCH v2] doc: update async enqueue API usage

Hi Xuan,

doc/guides/prog_guide/vhost_lib.rst the same vring with their own DMA virtual

Quoted 28 lines

-----Original Message-----
From: Ding, Xuan <[email protected]>
Sent: Tuesday, June 21, 2022 3:22 PM
To: [email protected]; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>
Subject: [PATCH v2] doc: update async enqueue API usage

From: Xuan Ding <[email protected]>

This patch updates the correct usage for async enqueue APIs. The rte_vhost_poll_enqueue_completed() needs to be called in time to notify the guest of completed packets and avoid packet loss.

---
v2:
* refine doc and commit log

---
 doc/guides/prog_guide/vhost_lib.rst | 8 ++++++++
 1 file changed, 8 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +0 -0
index cd3f6caa9a..628db3fc38 100644
@@ -480,6 +480,14 @@ the same vring with their own DMA virtual

channels.

doc/guides/prog_guide/vhost_lib.rst the same vring with their own DMA virtual channels. Besides, the number · line 490

Quoted 12 lines

Besides, the number
of DMA devices is limited. For the purpose of scaling, it's necessary
to support sharing DMA channels among vrings.

doc/guides/prog_guide/vhost_lib.rst +8 -0
+* Async enqueue API usage
+
+  In async enqueue path, rte_vhost_poll_enqueue_completed() needs to
+ be  called in time to notify the guest of DMA copy completed packets.
+  Moreover, calling rte_vhost_submit_enqueue_burst() all the time but
+ not poll completed will cause the DMA ring to be full, which will
+ result in packet loss eventually.
+

This subsection seems cleaner, could you also help to make 'Recommended IOVA mode in async datapath' as a sub-section?

Quoted 1 line

Another patch or not, either works for me.

I will send a separate patch to move 'Recommended IOVA mode in async datapath' as a sub-section.

Thanks,
Xuan

Quoted 9 lines

Thanks,
Chenbo

Quoted 5 lines

Recommended IOVA mode in async datapath
---------------------------------------

--
2.17.1

Reviewed-by on v2 1/1
#7 lore raw
In reply to [email protected]
Quoted 45 lines

-----Original Message-----
From: Ding, Xuan <[email protected]>
Sent: Tuesday, June 21, 2022 3:22 PM
To: [email protected]; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>
Subject: [PATCH v2] doc: update async enqueue API usage

From: Xuan Ding <[email protected]>

This patch updates the correct usage for async enqueue APIs. The rte_vhost_poll_enqueue_completed() needs to be called in time to notify the guest of completed packets and avoid packet loss.

---
v2:
* refine doc and commit log

---
 doc/guides/prog_guide/vhost_lib.rst | 8 ++++++++
 1 file changed, 8 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +0 -0
index cd3f6caa9a..628db3fc38 100644
@@ -480,6 +480,14 @@ the same vring with their own DMA virtual channels.

Besides, the number
of DMA devices is limited. For the purpose of scaling, it's necessary to support sharing DMA channels among vrings.

doc/guides/prog_guide/vhost_lib.rst +8 -1
+* Async enqueue API usage
+
+  In async enqueue path, rte_vhost_poll_enqueue_completed() needs to be
+  called in time to notify the guest of DMA copy completed packets.
+  Moreover, calling rte_vhost_submit_enqueue_burst() all the time but
+  not poll completed will cause the DMA ring to be full, which will
+  result in packet loss eventually.
+
 Recommended IOVA mode in async datapath
 ---------------------------------------
 
--
Quoted 1 line

2.17.1

#8 lore raw
In reply to [email protected]
Quoted 7 lines · On 6/21/22 09:21, [email protected] wrote:

From: Xuan Ding <[email protected]>

This patch updates the correct usage for async enqueue APIs. The rte_vhost_poll_enqueue_completed() needs to be called in time to notify the guest of completed packets and avoid packet loss.

Quoted 8 lines

---
v2:
* refine doc and commit log

---
  doc/guides/prog_guide/vhost_lib.rst | 8 ++++++++
  1 file changed, 8 insertions(+)

Applied to dpdk-next-virtio/main.

Thanks,
Maxime