Skip to contentSkip to conversation

vhost: add unsafe API to check inflight packets

PATCH Superseded Xuan Ding 3 participants 19 messages Sep 9, 2021 – Sep 28, 2021

lore
Share & tools
lore

Xuan Ding

author
#0 lore raw

PATCH 1/1 vhost: add unsafe API to check inflight packets

In async data path, when vring state changes, it is necessary to know the number of inflight packets in DMA engine. This patch provides a thread unsafe API to return the number of inflight packets without using any lock.

---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 25 +++++++++++++++++++++++++
 5 files changed, 52 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +5 -0
index 8874033165..b4b1134f54 100644
@@ -305,6 +305,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+    ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
doc/guides/rel_notes/release_21_11.rst +5 -0
index 675b573834..db080e9490 100644
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
lib/vhost/rte_vhost_async.h +14 -0
index b25ff446f7..0af414bf78 100644
@@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
lib/vhost/version.map +3 -0
index c92a9d4962..b150dc408d 100644
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
lib/vhost/vhost.c +25 -0
index 355ff37651..df96f84873 100644
@@ -1886,5 +1886,30 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
Signature · 1 line
2.17.1
#1 lore raw

Hi Xuan,

Quoted 34 lines

-----Original Message-----
From: Ding, Xuan <[email protected]>
Sent: Thursday, September 9, 2021 1:58 PM
To: [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Hu, Jiayu <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; Pai G, Sunil <[email protected]>; Ding, Xuan <[email protected]>
Subject: [PATCH] vhost: add unsafe API to check inflight packets

In async data path, when vring state changes, it is necessary to know the number of inflight packets in DMA engine. This patch provides a thread unsafe API to return the number of inflight packets without using any lock.

---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 25 +++++++++++++++++++++++++
 5 files changed, 52 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +0 -0
index 8874033165..b4b1134f54 100644
@@ -305,6 +305,11 @@ The following is an overview of some key Vhost API

functions:
This function returns the amount of in-flight packets for the vhost queue using async acceleration.

doc/guides/prog_guide/vhost_lib.rst The following is an overview of some key Vhost API functions: · line 313

Quoted 6 lines
doc/guides/prog_guide/vhost_lib.rst +5 -0
+    ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``

This does not align with others. Please check.

Quoted 9 lines

Clear inflight packets which are submitted to DMA engine in vhost async
data

doc/guides/rel_notes/release_21_11.rst +0 -0
index 675b573834..db080e9490 100644

doc/guides/rel_notes/release_21_11.rst New Features · line 62

Quoted 8 lines
doc/guides/rel_notes/release_21_11.rst +5 -0
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path.
+

Please add 'without lock' or something similar as we already have a lock version.

Quoted 10 lines
doc/guides/rel_notes/release_21_11.rst +0 -0
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
lib/vhost/rte_vhost_async.h +0 -0
index b25ff446f7..0af414bf78 100644
@@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid,
 uint16_t queue_id,
 __rte_experimental

lib/vhost/rte_vhost_async.h uint16_t rte_vhost_poll_enqueue_completed(int vid, · line 257

Quoted 10 lines
lib/vhost/rte_vhost_async.h +8 -0
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data

You can also check dequeue inflight packets, right?

Quoted 26 lines
lib/vhost/rte_vhost_async.h +6 -0
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for

* a specific vhost device queue. Packets which are inflight will be

lib/vhost/version.map +3 -0
index c92a9d4962..b150dc408d 100644
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
lib/vhost/vhost.c +0 -0
index 355ff37651..df96f84873 100644

lib/vhost/vhost.c int rte_vhost_async_get_inflight(int vid, uint16_t · line 1890

Quoted 5 lines
lib/vhost/vhost.c +1 -0
@@ -1886,5 +1886,30 @@ int rte_vhost_async_get_inflight(int vid, uint16_t
 queue_id)
 	return ret;
 }
 
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)

According to DPDK coding style, return type and func name should be in different lines. I notice there are also some left to clean. May clean in another patch.

Thanks,
Chenbo

Quoted 28 lines
lib/vhost/vhost.c +24 -0
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);

RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
--
2.17.1

author
#2 lore raw

Hi Chenbo,

Quoted 46 lines

-----Original Message-----
From: Xia, Chenbo <[email protected]>
Sent: Wednesday, September 15, 2021 2:49 PM
To: Ding, Xuan <[email protected]>; [email protected]; [email protected]
Cc: Hu, Jiayu <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; Pai G, Sunil <[email protected]>
Subject: RE: [PATCH] vhost: add unsafe API to check inflight packets

Hi Xuan,

Quoted 7 lines

-----Original Message-----
From: Ding, Xuan <[email protected]>
Sent: Thursday, September 9, 2021 1:58 PM
To: [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Hu, Jiayu <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; Pai G, Sunil <[email protected]>; Ding,

Xuan

Quoted 27 lines

<[email protected]>
Subject: [PATCH] vhost: add unsafe API to check inflight packets

In async data path, when vring state changes, it is necessary to know the number of inflight packets in DMA engine. This patch provides a thread unsafe API to return the number of inflight packets without using any lock.

---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 25 +++++++++++++++++++++++++
 5 files changed, 52 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +0 -0
index 8874033165..b4b1134f54 100644
@@ -305,6 +305,11 @@ The following is an overview of some key Vhost API

functions:
This function returns the amount of in-flight packets for the vhost queue using async acceleration.

Quoted 8 lines

doc/guides/prog_guide/vhost_lib.rst The following is an overview of some key Vhost API functions: · line 313

Quoted 6 lines
doc/guides/prog_guide/vhost_lib.rst +5 -0
+    ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``

This does not align with others. Please check.

Thanks, will fix it in next version.

Quoted 13 lines
Quoted 13 lines

Clear inflight packets which are submitted to DMA engine in vhost async
data

doc/guides/rel_notes/release_21_11.rst +0 -0
index 675b573834..db080e9490 100644
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
Quoted 7 lines

doc/guides/rel_notes/release_21_11.rst New Features · line 62

Quoted 5 lines
doc/guides/rel_notes/release_21_11.rst +5 -0
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path.
+

Please add 'without lock' or something similar as we already have a lock version.

You are right, add "without lock" is more accuracy.

Quoted 13 lines
Quoted 13 lines
doc/guides/rel_notes/release_21_11.rst +0 -0
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
lib/vhost/rte_vhost_async.h +0 -0
index b25ff446f7..0af414bf78 100644
@@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid,
 uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
Quoted 10 lines

lib/vhost/rte_vhost_async.h uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id, · line 256

Quoted 8 lines
lib/vhost/rte_vhost_async.h +8 -0
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data

You can also check dequeue inflight packets, right?

Yes, this API applies to both enqueue and dequeue directions.

Quoted 31 lines
Quoted 31 lines
lib/vhost/rte_vhost_async.h +6 -0
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for

* a specific vhost device queue. Packets which are inflight will be

lib/vhost/version.map +3 -0
index c92a9d4962..b150dc408d 100644
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
lib/vhost/vhost.c +0 -0
index 355ff37651..df96f84873 100644
@@ -1886,5 +1886,30 @@ int rte_vhost_async_get_inflight(int vid, uint16_t
 queue_id)
 	return ret;
 }
 
Quoted 5 lines

lib/vhost/vhost.c int rte_vhost_async_get_inflight(int vid, uint16_t queue_id) · line 1889

Quoted 1 line
lib/vhost/vhost.c +1 -0
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)

According to DPDK coding style, return type and func name should be in different
lines. I notice there are also some left to clean. May clean in another patch.

Thanks, notice the rte_vhost_async_get_inflight() API also has this problem. Will fix both in next version.

Regards,
Xuan

Quoted 32 lines

Thanks,
Chenbo

Quoted 28 lines
lib/vhost/vhost.c +24 -0
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);

RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
--
2.17.1

author
#3 lore raw

PATCH v2 1/1 vhost: add unsafe API to check inflight packets

In reply to Xuan Ding

In async data path, when vring state changes, it is necessary to know the number of inflight packets in DMA engine. This patch provides a thread unsafe API to return the number of inflight packets without using any lock.

---

v2:
* Fixed some format issues.

---
 doc/guides/prog_guide/vhost_lib.rst    |  5 ++++
 doc/guides/rel_notes/release_21_11.rst |  5 ++++
 lib/vhost/rte_vhost_async.h            | 14 +++++++++
 lib/vhost/version.map                  |  3 ++
 lib/vhost/vhost.c                      | 41 ++++++++++++++++++++++----
 5 files changed, 63 insertions(+), 5 deletions(-)
doc/guides/prog_guide/vhost_lib.rst +5 -0
index 8874033165..0c4fb9ea91 100644
@@ -305,6 +305,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+* ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
doc/guides/rel_notes/release_21_11.rst +5 -0
index 675b573834..c08e2b80c4 100644
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path without lock.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
lib/vhost/rte_vhost_async.h +14 -0
index b25ff446f7..0af414bf78 100644
@@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
lib/vhost/version.map +3 -0
index c92a9d4962..b150dc408d 100644
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
lib/vhost/vhost.c +36 -5

76 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#4 lore raw

PATCH v3 1/1 vhost: add unsafe API to check inflight packets

In reply to Xuan Ding

In async data path, when vring state changes, it is necessary to know the number of inflight packets in DMA engine. This patch provides a thread unsafe API to return the number of inflight packets without using any lock.

---
v3:
* Fixed one typo.
* Revised the doc to be more accuracy.

v2:
* Fixed some format issues.

---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +5 -0
index 8874033165..0c4fb9ea91 100644
@@ -305,6 +305,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+* ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
doc/guides/rel_notes/release_21_11.rst +5 -0
index 675b573834..c9814c68df 100644
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path without using lock.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
lib/vhost/rte_vhost_async.h +14 -0
index b25ff446f7..0af414bf78 100644
@@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
lib/vhost/version.map +3 -0
index c92a9d4962..b150dc408d 100644
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
lib/vhost/vhost.c +26 -0
index 355ff37651..24ae1025ea 100644
@@ -1886,5 +1886,31 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int
+rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
Signature · 1 line
2.17.1
#5 lore raw
Quoted 13 lines · Xuan Ding

In async data path, when vring state changes, it is necessary to know the number of inflight packets in DMA engine. This patch provides a thread unsafe API to return the number of inflight packets without using any lock.

---
v3:
* Fixed one typo.
* Revised the doc to be more accuracy.

v2:
* Fixed some format issues.

Quoted 8 lines
---
  doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
  doc/guides/rel_notes/release_21_11.rst |  5 +++++
  lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
  lib/vhost/version.map                  |  3 +++
  lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
  5 files changed, 53 insertions(+)

<snip>

Should there be a change to the vring_state_changed() in vhost example app to go along with this patch? It would help to understand the operation if this API was used.

author
#6 lore raw

Hi,

Quoted 34 lines

-----Original Message-----
From: Kevin Traynor <[email protected]>
Sent: Wednesday, September 22, 2021 1:28 AM
To: Ding, Xuan <[email protected]>; [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Hu, Jiayu <[email protected]>; Jiang, Cheng1 <[email protected]>; Richardson, Bruce <[email protected]>; Pai G, Sunil <[email protected]>; Yang, YvonneX <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v3] vhost: add unsafe API to check inflight packets

Quoted 13 lines · Xuan Ding

In async data path, when vring state changes, it is necessary to know the number of inflight packets in DMA engine. This patch provides a thread unsafe API to return the number of inflight packets without using any lock.

---
v3:
* Fixed one typo.
* Revised the doc to be more accuracy.

v2:
* Fixed some format issues.

Quoted 8 lines
---
  doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
  doc/guides/rel_notes/release_21_11.rst |  5 +++++
  lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
  lib/vhost/version.map                  |  3 +++
  lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
  5 files changed, 53 insertions(+)
Quoted 5 lines

<snip>

Should there be a change to the vring_state_changed() in vhost example app to go along with this patch? It would help to understand the operation if this API was used.

Thanks Kevin, will add it in vhost example in next version.

Regards,
Xuan

author
#7 lore raw

v4 cover add unsafe API to get inflight packets

In reply to Xuan Ding

This patchset introduces an unsafe API to get the number of inflight packets in DMA engine in some situations. Like vring state changes or device is destroyed. Compared with rte_vhost_async_get_inflight(), this is a lock free version.

v4:
* Added use case for API in vhost example.

v3:
* Fixed one typo.
* Revised the doc to be more accuracy.

v2:
* Fixed some format issues.

Xuan Ding (2):
vhost: add unsafe API to check inflight packets examples/vhost: use API to check inflight packets

 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 examples/vhost/main.c                  | 26 +++++++++++---------------
 examples/vhost/main.h                  |  1 -
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 7 files changed, 64 insertions(+), 16 deletions(-)
Signature · 1 line
2.17.1
author
#8 lore raw

PATCH v4 1/2 vhost: add unsafe API to check inflight packets

In async data path, when vring state changes or device is destroyed, it is necessary to know the number of inflight packets in DMA engine. This patch provides a thread unsafe API to return the number of inflight packets for a vhost queue without using any lock.

---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +5 -0
index 8874033165..0c4fb9ea91 100644
@@ -305,6 +305,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+* ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
doc/guides/rel_notes/release_21_11.rst +5 -0
index 675b573834..c9814c68df 100644
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path without using lock.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
lib/vhost/rte_vhost_async.h +14 -0
index b25ff446f7..0af414bf78 100644
@@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
lib/vhost/version.map +3 -0
index c92a9d4962..b150dc408d 100644
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
lib/vhost/vhost.c +26 -0
index 355ff37651..24ae1025ea 100644
@@ -1886,5 +1886,31 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int
+rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
Signature · 1 line
2.17.1
author
#9 lore raw

PATCH v4 2/2 examples/vhost: use API to check inflight packets

In reply to Xuan Ding

In async data path, call rte_vhost_async_get_inflight_thread_unsafe() API to directly return the number of inflight packets instead of maintaining a local variable.

---
 examples/vhost/main.c | 26 +++++++++++---------------
 examples/vhost/main.h |  1 -
 2 files changed, 11 insertions(+), 16 deletions(-)
examples/vhost/main.c +11 -15
index bc3d71c898..f0b74b5086 100644
@@ -851,11 +851,8 @@ complete_async_pkts(struct vhost_dev *vdev)
 
 	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
 					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
-	if (complete_count) {
+	if (complete_count)
 		free_pkts(p_cpl, complete_count);
-		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count, __ATOMIC_SEQ_CST);
-	}
-
 }
 
 static __rte_always_inline void
@@ -898,7 +895,6 @@ drain_vhost(struct vhost_dev *vdev)
 		complete_async_pkts(vdev);
 		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ,
 					m, nr_xmit, m_cpu_cpl, &cpu_cpl_nr);
-		__atomic_add_fetch(&vdev->pkts_inflight, ret - cpu_cpl_nr, __ATOMIC_SEQ_CST);
 
 		if (cpu_cpl_nr)
 			free_pkts(m_cpu_cpl, cpu_cpl_nr);
@@ -1230,8 +1226,6 @@ drain_eth_rx(struct vhost_dev *vdev)
 		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
 					VIRTIO_RXQ, pkts, rx_count,
 					m_cpu_cpl, &cpu_cpl_nr);
-		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count - cpu_cpl_nr,
-					__ATOMIC_SEQ_CST);
 
 		if (cpu_cpl_nr)
 			free_pkts(m_cpu_cpl, cpu_cpl_nr);
@@ -1360,6 +1354,7 @@ destroy_device(int vid)
 	struct vhost_dev *vdev = NULL;
 	int lcore;
 	uint16_t i;
+	int pkts_inflight;
 
 	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
 		if (vdev->vid == vid)
@@ -1406,13 +1401,13 @@ destroy_device(int vid)
 
 	if (async_vhost_driver) {
 		uint16_t n_pkt = 0;
-		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, VIRTIO_RXQ);
+		struct rte_mbuf *m_cpl[pkts_inflight];
 
-		while (vdev->pkts_inflight) {
+		while (pkts_inflight) {
 			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, VIRTIO_RXQ,
-						m_cpl, vdev->pkts_inflight);
+						m_cpl, pkts_inflight);
 			free_pkts(m_cpl, n_pkt);
-			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
 		}
 
 		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
@@ -1509,6 +1504,7 @@ static int
 vring_state_changed(int vid, uint16_t queue_id, int enable)
 {
 	struct vhost_dev *vdev = NULL;
+	int pkts_inflight;
 
 	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
 		if (vdev->vid == vid)
@@ -1523,13 +1519,13 @@ vring_state_changed(int vid, uint16_t queue_id, int enable)
 	if (async_vhost_driver) {
 		if (!enable) {
 			uint16_t n_pkt = 0;
-			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+			pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);
+			struct rte_mbuf *m_cpl[pkts_inflight];
 
-			while (vdev->pkts_inflight) {
+			while (pkts_inflight) {
 				n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, queue_id,
-							m_cpl, vdev->pkts_inflight);
+							m_cpl, pkts_inflight);
 				free_pkts(m_cpl, n_pkt);
-				__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
 			}
 		}
 	}
examples/vhost/main.h +0 -1
index e7b1ac60a6..0ccdce4b4a 100644
@@ -51,7 +51,6 @@ struct vhost_dev {
 	uint64_t features;
 	size_t hdr_len;
 	uint16_t nr_vrings;
-	uint16_t pkts_inflight;
 	struct rte_vhost_memory *mem;
 	struct device_statistics stats;
 	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
Signature · 1 line
2.17.1
author
#10 lore raw

v5 cover add unsafe API to get inflight packets

In reply to Xuan Ding

This patchset introduces an unsafe API to get the number of inflight packets in DMA engine in some situations. Like vring state changes or device is destroyed. Compared with rte_vhost_async_get_inflight(), this is a lock free version.

v5:
* Rebased to the lastest branch.

v4:
* Added use case for API in vhost example.

v3:
* Fixed one typo.
* Revised the doc to be more accuracy.

v2:
* Fixed some format issues.

Xuan Ding (2):
vhost: add unsafe API to check inflight packets examples/vhost: use API to check inflight packets

 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 examples/vhost/main.c                  | 25 +++++++++++--------------
 examples/vhost/main.h                  |  1 -
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 7 files changed, 64 insertions(+), 15 deletions(-)
Signature · 1 line
2.17.1
author
#11 lore raw

PATCH v5 1/2 vhost: add unsafe API to check inflight packets

In async data path, when vring state changes or device is destroyed, it is necessary to know the number of inflight packets in DMA engine. This patch provides a thread unsafe API to return the number of inflight packets for a vhost queue without using any lock.

---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +5 -0
index 171e0096f6..73a4e24640 100644
@@ -298,6 +298,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+* ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
doc/guides/rel_notes/release_21_11.rst +5 -0
index ad7c1afec0..fd821b29d3 100644
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path without using lock.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
lib/vhost/rte_vhost_async.h +14 -0
index ad71555a7f..1f0708378b 100644
@@ -234,6 +234,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
lib/vhost/version.map +3 -0
index 8ebde3f694..ed6cb53a99 100644
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
lib/vhost/vhost.c +26 -0
index 996287c3c2..c16c6bc2bb 100644
@@ -1884,5 +1884,31 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int
+rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
Signature · 1 line
2.17.1
author
#12 lore raw

PATCH v5 2/2 examples/vhost: use API to check inflight packets

In reply to Xuan Ding

In async data path, call rte_vhost_async_get_inflight_thread_unsafe() API to directly return the number of inflight packets instead of maintaining a local variable.

---
 examples/vhost/main.c | 25 +++++++++++--------------
 examples/vhost/main.h |  1 -
 2 files changed, 11 insertions(+), 15 deletions(-)
examples/vhost/main.c +11 -14
index d0bf1f31e3..3faac6d053 100644
@@ -842,11 +842,8 @@ complete_async_pkts(struct vhost_dev *vdev)
 
 	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
 					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
-	if (complete_count) {
+	if (complete_count)
 		free_pkts(p_cpl, complete_count);
-		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count, __ATOMIC_SEQ_CST);
-	}
-
 }
 
 static __rte_always_inline void
@@ -886,7 +883,6 @@ drain_vhost(struct vhost_dev *vdev)
 
 		complete_async_pkts(vdev);
 		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ, m, nr_xmit);
-		__atomic_add_fetch(&vdev->pkts_inflight, ret, __ATOMIC_SEQ_CST);
 
 		enqueue_fail = nr_xmit - ret;
 		if (enqueue_fail)
@@ -1212,7 +1208,6 @@ drain_eth_rx(struct vhost_dev *vdev)
 		complete_async_pkts(vdev);
 		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
 					VIRTIO_RXQ, pkts, rx_count);
-		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count, __ATOMIC_SEQ_CST);
 
 		enqueue_fail = rx_count - enqueue_count;
 		if (enqueue_fail)
@@ -1338,6 +1333,7 @@ destroy_device(int vid)
 	struct vhost_dev *vdev = NULL;
 	int lcore;
 	uint16_t i;
+	int pkts_inflight;
 
 	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
 		if (vdev->vid == vid)
@@ -1384,13 +1380,13 @@ destroy_device(int vid)
 
 	if (async_vhost_driver) {
 		uint16_t n_pkt = 0;
-		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, VIRTIO_RXQ);
+		struct rte_mbuf *m_cpl[pkts_inflight];
 
-		while (vdev->pkts_inflight) {
+		while (pkts_inflight) {
 			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, VIRTIO_RXQ,
-						m_cpl, vdev->pkts_inflight);
+						m_cpl, pkts_inflight);
 			free_pkts(m_cpl, n_pkt);
-			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
 		}
 
 		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
@@ -1486,6 +1482,7 @@ static int
 vring_state_changed(int vid, uint16_t queue_id, int enable)
 {
 	struct vhost_dev *vdev = NULL;
+	int pkts_inflight;
 
 	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
 		if (vdev->vid == vid)
@@ -1500,13 +1497,13 @@ vring_state_changed(int vid, uint16_t queue_id, int enable)
 	if (async_vhost_driver) {
 		if (!enable) {
 			uint16_t n_pkt = 0;
-			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+			pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);
+			struct rte_mbuf *m_cpl[pkts_inflight];
 
-			while (vdev->pkts_inflight) {
+			while (pkts_inflight) {
 				n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, queue_id,
-							m_cpl, vdev->pkts_inflight);
+							m_cpl, pkts_inflight);
 				free_pkts(m_cpl, n_pkt);
-				__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
 			}
 		}
 	}
examples/vhost/main.h +0 -1
index e7b1ac60a6..0ccdce4b4a 100644
@@ -51,7 +51,6 @@ struct vhost_dev {
 	uint64_t features;
 	size_t hdr_len;
 	uint16_t nr_vrings;
-	uint16_t pkts_inflight;
 	struct rte_vhost_memory *mem;
 	struct device_statistics stats;
 	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
Signature · 1 line
2.17.1
#13 lore raw

Re: [dpdk-dev] [PATCH v5 2/2] examples/vhost: use API to check inflight packets

Quoted 44 lines · Xuan Ding

In async data path, call rte_vhost_async_get_inflight_thread_unsafe() API to directly return the number of inflight packets instead of maintaining a local variable.

---
  examples/vhost/main.c | 25 +++++++++++--------------
  examples/vhost/main.h |  1 -
  2 files changed, 11 insertions(+), 15 deletions(-)
examples/vhost/main.c +1 -6
index d0bf1f31e3..3faac6d053 100644
@@ -842,11 +842,8 @@ complete_async_pkts(struct vhost_dev *vdev)
 
  	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
  					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
-	if (complete_count) {
+	if (complete_count)
  		free_pkts(p_cpl, complete_count);
-		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count, __ATOMIC_SEQ_CST);
-	}
-
  }
 
  static __rte_always_inline void
@@ -886,7 +883,6 @@ drain_vhost(struct vhost_dev *vdev)
 
  		complete_async_pkts(vdev);
  		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ, m, nr_xmit);
-		__atomic_add_fetch(&vdev->pkts_inflight, ret, __ATOMIC_SEQ_CST);
 
  		enqueue_fail = nr_xmit - ret;
  		if (enqueue_fail)
@@ -1212,7 +1208,6 @@ drain_eth_rx(struct vhost_dev *vdev)
  		complete_async_pkts(vdev);
  		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
  					VIRTIO_RXQ, pkts, rx_count);
-		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count, __ATOMIC_SEQ_CST);
 
  		enqueue_fail = rx_count - enqueue_count;
  		if (enqueue_fail)

examples/vhost/main.c destroy_device(int vid) · line 1337

Quoted 5 lines
examples/vhost/main.c +1 -0
@@ -1338,6 +1333,7 @@ destroy_device(int vid)
  	struct vhost_dev *vdev = NULL;
  	int lcore;
  	uint16_t i;
 
+	int pkts_inflight;

You can move this down to the block it is used in

Quoted 8 lines
examples/vhost/main.c +0 -1
  	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
  		if (vdev->vid == vid)
@@ -1384,13 +1380,13 @@ destroy_device(int vid)
 
  	if (async_vhost_driver) {
  		uint16_t n_pkt = 0;
-		struct rte_mbuf *m_cpl[vdev->pkts_inflight];

examples/vhost/main.c destroy_device(int vid) · line 1393

Quoted 10 lines
examples/vhost/main.c +4 -3
+		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, VIRTIO_RXQ);
+		struct rte_mbuf *m_cpl[pkts_inflight];
 
-		while (vdev->pkts_inflight) {
+		while (pkts_inflight) {
  			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, VIRTIO_RXQ,
-						m_cpl, vdev->pkts_inflight);
+						m_cpl, pkts_inflight);
  			free_pkts(m_cpl, n_pkt);
-			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);

This is an infinite loop if there are pkts_inflight, need to recheck pkts_inflight in the loop.

Quoted 16 lines
examples/vhost/main.c +1 -1
  		}
 
  		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
@@ -1486,6 +1482,7 @@ static int
  vring_state_changed(int vid, uint16_t queue_id, int enable)
  {
  	struct vhost_dev *vdev = NULL;
+	int pkts_inflight;
 
  	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
  		if (vdev->vid == vid)
@@ -1500,13 +1497,13 @@ vring_state_changed(int vid, uint16_t queue_id, int enable)
  	if (async_vhost_driver) {
  		if (!enable) {
  			uint16_t n_pkt = 0;
-			struct rte_mbuf *m_cpl[vdev->pkts_inflight];

examples/vhost/main.c vring_state_changed(int vid, uint16_t queue_id, int enable) · line 1509

Quoted 10 lines
examples/vhost/main.c +4 -3
+			pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);
+			struct rte_mbuf *m_cpl[pkts_inflight];
 
-			while (vdev->pkts_inflight) {
+			while (pkts_inflight) {
  				n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, queue_id,
-							m_cpl, vdev->pkts_inflight);
+							m_cpl, pkts_inflight);
  				free_pkts(m_cpl, n_pkt);
-				__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);

Same comments as destroy_device

Quoted 16 lines
examples/vhost/main.c +0 -0
  			}
  		}
  	}
examples/vhost/main.h +0 -1
index e7b1ac60a6..0ccdce4b4a 100644
@@ -51,7 +51,6 @@ struct vhost_dev {
  	uint64_t features;
  	size_t hdr_len;
  	uint16_t nr_vrings;
-	uint16_t pkts_inflight;
  	struct rte_vhost_memory *mem;
  	struct device_statistics stats;
  	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
author
#14 lore raw

Re: [dpdk-dev] [PATCH v5 2/2] examples/vhost: use API to check inflight packets

Hi Kevin,

Quoted 64 lines

-----Original Message-----
From: Kevin Traynor <[email protected]>
Sent: Tuesday, September 28, 2021 5:18 PM
To: Ding, Xuan <[email protected]>; [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Hu, Jiayu <[email protected]>; Jiang, Cheng1 <[email protected]>; Richardson, Bruce <[email protected]>; Pai G, Sunil <[email protected]>; Wang, Yinan <[email protected]>; Yang, YvonneX <[email protected]>
Subject: Re: [PATCH v5 2/2] examples/vhost: use API to check inflight packets

Quoted 15 lines · Xuan Ding

In async data path, call rte_vhost_async_get_inflight_thread_unsafe() API to directly return the number of inflight packets instead of maintaining a local variable.

---
  examples/vhost/main.c | 25 +++++++++++--------------
  examples/vhost/main.h |  1 -
  2 files changed, 11 insertions(+), 15 deletions(-)
examples/vhost/main.c +0 -0
index d0bf1f31e3..3faac6d053 100644

examples/vhost/main.c complete_async_pkts(struct vhost_dev *vdev) · line 847

Quoted 7 lines
examples/vhost/main.c +1 -2
@@ -842,11 +842,8 @@ complete_async_pkts(struct vhost_dev *vdev)
 
  	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
  					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
-	if (complete_count) {
+	if (complete_count)
  		free_pkts(p_cpl, complete_count);
-		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count,

__ATOMIC_SEQ_CST);

examples/vhost/main.c drain_vhost(struct vhost_dev *vdev) · line 885

Quoted 9 lines
examples/vhost/main.c +0 -2
-	}
-
  }
 
  static __rte_always_inline void
@@ -886,7 +883,6 @@ drain_vhost(struct vhost_dev *vdev)
 
  		complete_async_pkts(vdev);
  		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ,

m, nr_xmit);

examples/vhost/main.c drain_vhost(struct vhost_dev *vdev) · line 889

Quoted 1 line
examples/vhost/main.c +0 -1
-		__atomic_add_fetch(&vdev->pkts_inflight, ret,

__ATOMIC_SEQ_CST);

examples/vhost/main.c drain_eth_rx(struct vhost_dev *vdev) · line 1215

Quoted 8 lines
examples/vhost/main.c +0 -1
  		enqueue_fail = nr_xmit - ret;
  		if (enqueue_fail)
@@ -1212,7 +1208,6 @@ drain_eth_rx(struct vhost_dev *vdev)
  		complete_async_pkts(vdev);
  		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
  					VIRTIO_RXQ, pkts, rx_count);
-		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count,

__ATOMIC_SEQ_CST);

Quoted 7 lines
examples/vhost/main.c +0 -0
  		enqueue_fail = rx_count - enqueue_count;
  		if (enqueue_fail)
@@ -1338,6 +1333,7 @@ destroy_device(int vid)
  	struct vhost_dev *vdev = NULL;
  	int lcore;
  	uint16_t i;
Quoted 3 lines

examples/vhost/main.c destroy_device(int vid) · line 1336

Quoted 1 line
examples/vhost/main.c +1 -0
+	int pkts_inflight;

You can move this down to the block it is used in

Thanks for the suggestion.
I consider calling the unsafe API in while (condition), and there is no need to define this variable.

Quoted 17 lines

examples/vhost/main.c destroy_device(int vid) · line 1383

Quoted 10 lines
examples/vhost/main.c +1 -1
  	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
  		if (vdev->vid == vid)
@@ -1384,13 +1380,13 @@ destroy_device(int vid)
 
  	if (async_vhost_driver) {
  		uint16_t n_pkt = 0;
-		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid,

VIRTIO_RXQ);

examples/vhost/main.c destroy_device(int vid)

Quoted 5 lines
examples/vhost/main.c +2 -1
+		struct rte_mbuf *m_cpl[pkts_inflight];
 
-		while (vdev->pkts_inflight) {
+		while (pkts_inflight) {
  			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid,

VIRTIO_RXQ,

Quoted 8 lines

examples/vhost/main.c destroy_device(int vid)

Quoted 4 lines
examples/vhost/main.c +1 -2
-						m_cpl, vdev->pkts_inflight);
+						m_cpl, pkts_inflight);
  			free_pkts(m_cpl, n_pkt);
-			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt,

__ATOMIC_SEQ_CST);

This is an infinite loop if there are pkts_inflight, need to recheck pkts_inflight in the loop.

Thanks for the catch, will call the unsafe API directly in the while (condition).

Quoted 26 lines

examples/vhost/main.c vring_state_changed(int vid, uint16_t queue_id,

Quoted 13 lines
examples/vhost/main.c +1 -0
  		}
 
  		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
@@ -1486,6 +1482,7 @@ static int
  vring_state_changed(int vid, uint16_t queue_id, int enable)
  {
  	struct vhost_dev *vdev = NULL;
+	int pkts_inflight;
 
  	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
  		if (vdev->vid == vid)
@@ -1500,13 +1497,13 @@ vring_state_changed(int vid, uint16_t queue_id,

int enable)

examples/vhost/main.c vring_state_changed(int vid, uint16_t queue_id, int enable)

Quoted 5 lines
examples/vhost/main.c +1 -1
  	if (async_vhost_driver) {
  		if (!enable) {
  			uint16_t n_pkt = 0;
-			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+			pkts_inflight =

rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);

examples/vhost/main.c destroy_device(int vid)

Quoted 5 lines
examples/vhost/main.c +2 -1
+			struct rte_mbuf *m_cpl[pkts_inflight];
 
-			while (vdev->pkts_inflight) {
+			while (pkts_inflight) {
  				n_pkt =

rte_vhost_clear_queue_thread_unsafe(vid, queue_id,

Quoted 8 lines

examples/vhost/main.c vring_state_changed(int vid, uint16_t queue_id, · line 1510

Quoted 5 lines
examples/vhost/main.c +1 -2
-							m_cpl, vdev-
 pkts_inflight);
+							m_cpl, pkts_inflight);
  				free_pkts(m_cpl, n_pkt);
-				__atomic_sub_fetch(&vdev->pkts_inflight,

n_pkt, __ATOMIC_SEQ_CST);

Same comments as destroy_device

Same as vring_state_changed.

Regards,
Xuan

Quoted 17 lines
Quoted 17 lines
examples/vhost/main.c +0 -0
  			}
  		}
  	}
examples/vhost/main.h +0 -1
index e7b1ac60a6..0ccdce4b4a 100644
@@ -51,7 +51,6 @@ struct vhost_dev {
  	uint64_t features;
  	size_t hdr_len;
  	uint16_t nr_vrings;
-	uint16_t pkts_inflight;
  	struct rte_vhost_memory *mem;
  	struct device_statistics stats;
  	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
author
#15 lore raw

Re: [dpdk-dev] [PATCH v5 2/2] examples/vhost: use API to check inflight packets

Hi,

Quoted 109 lines

-----Original Message-----
From: Ding, Xuan
Sent: Tuesday, September 28, 2021 7:51 PM
To: Kevin Traynor <[email protected]>; [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Hu, Jiayu <[email protected]>; Jiang, Cheng1 <[email protected]>; Richardson, Bruce <[email protected]>; Pai G, Sunil <[email protected]>; Wang, Yinan <[email protected]>; Yang, YvonneX <[email protected]>
Subject: RE: [PATCH v5 2/2] examples/vhost: use API to check inflight packets

Hi Kevin,

Quoted 8 lines

-----Original Message-----
From: Kevin Traynor <[email protected]>
Sent: Tuesday, September 28, 2021 5:18 PM
To: Ding, Xuan <[email protected]>; [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Hu, Jiayu <[email protected]>; Jiang, Cheng1 <[email protected]>; Richardson, Bruce <[email protected]>; Pai G, Sunil <[email protected]>; Wang, Yinan <[email protected]>; Yang,

YvonneX

Quoted 56 lines

<[email protected]>
Subject: Re: [PATCH v5 2/2] examples/vhost: use API to check inflight packets

Quoted 15 lines · Xuan Ding

In async data path, call rte_vhost_async_get_inflight_thread_unsafe() API to directly return the number of inflight packets instead of maintaining a local variable.

---
  examples/vhost/main.c | 25 +++++++++++--------------
  examples/vhost/main.h |  1 -
  2 files changed, 11 insertions(+), 15 deletions(-)
examples/vhost/main.c +0 -0
index d0bf1f31e3..3faac6d053 100644

examples/vhost/main.c complete_async_pkts(struct vhost_dev *vdev) · line 847

Quoted 7 lines
examples/vhost/main.c +1 -2
@@ -842,11 +842,8 @@ complete_async_pkts(struct vhost_dev *vdev)
 
  	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
  					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
-	if (complete_count) {
+	if (complete_count)
  		free_pkts(p_cpl, complete_count);
-		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count,

__ATOMIC_SEQ_CST);

examples/vhost/main.c drain_vhost(struct vhost_dev *vdev) · line 885

Quoted 9 lines
examples/vhost/main.c +0 -2
-	}
-
  }
 
  static __rte_always_inline void
@@ -886,7 +883,6 @@ drain_vhost(struct vhost_dev *vdev)
 
  		complete_async_pkts(vdev);
  		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ,

m, nr_xmit);

examples/vhost/main.c drain_vhost(struct vhost_dev *vdev) · line 889

Quoted 1 line
examples/vhost/main.c +0 -1
-		__atomic_add_fetch(&vdev->pkts_inflight, ret,

__ATOMIC_SEQ_CST);

examples/vhost/main.c drain_eth_rx(struct vhost_dev *vdev) · line 1215

Quoted 8 lines
examples/vhost/main.c +0 -1
  		enqueue_fail = nr_xmit - ret;
  		if (enqueue_fail)
@@ -1212,7 +1208,6 @@ drain_eth_rx(struct vhost_dev *vdev)
  		complete_async_pkts(vdev);
  		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
  					VIRTIO_RXQ, pkts, rx_count);
-		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count,

__ATOMIC_SEQ_CST);

Quoted 7 lines
examples/vhost/main.c +0 -0
  		enqueue_fail = rx_count - enqueue_count;
  		if (enqueue_fail)
@@ -1338,6 +1333,7 @@ destroy_device(int vid)
  	struct vhost_dev *vdev = NULL;
  	int lcore;
  	uint16_t i;
Quoted 3 lines

examples/vhost/main.c destroy_device(int vid) · line 1336

Quoted 1 line
examples/vhost/main.c +1 -0
+	int pkts_inflight;

You can move this down to the block it is used in

Thanks for the suggestion.
I consider calling the unsafe API in while (condition), and there is no need to define this variable.

Quoted 23 lines

examples/vhost/main.c destroy_device(int vid) · line 1383

Quoted 10 lines
examples/vhost/main.c +1 -1
  	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
  		if (vdev->vid == vid)
@@ -1384,13 +1380,13 @@ destroy_device(int vid)
 
  	if (async_vhost_driver) {
  		uint16_t n_pkt = 0;
-		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid,

VIRTIO_RXQ);

examples/vhost/main.c destroy_device(int vid)

Quoted 5 lines
examples/vhost/main.c +2 -1
+		struct rte_mbuf *m_cpl[pkts_inflight];
 
-		while (vdev->pkts_inflight) {
+		while (pkts_inflight) {
  			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid,

VIRTIO_RXQ,

examples/vhost/main.c destroy_device(int vid)

Quoted 4 lines
examples/vhost/main.c +1 -2
-						m_cpl, vdev->pkts_inflight);
+						m_cpl, pkts_inflight);
  			free_pkts(m_cpl, n_pkt);
-			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt,

__ATOMIC_SEQ_CST);

Quoted 4 lines
Quoted 2 lines

This is an infinite loop if there are pkts_inflight, need to recheck pkts_inflight in the loop.

Thanks for the catch, will call the unsafe API directly in the while (condition).

Sorry for replying myself, as rte_mbuf *m_cpl also needs pkts_inflight here. Will follow your suggestion, see next version, thanks!

Regards,
Xuan

Quoted 58 lines
Quoted 27 lines

examples/vhost/main.c vring_state_changed(int vid, uint16_t queue_id,

Quoted 14 lines
examples/vhost/main.c +1 -0
  		}
 
  		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
@@ -1486,6 +1482,7 @@ static int
  vring_state_changed(int vid, uint16_t queue_id, int enable)
  {
  	struct vhost_dev *vdev = NULL;
+	int pkts_inflight;
 
  	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
  		if (vdev->vid == vid)
@@ -1500,13 +1497,13 @@ vring_state_changed(int vid, uint16_t queue_id,

int enable)

examples/vhost/main.c vring_state_changed(int vid, uint16_t queue_id, int enable)

Quoted 5 lines
examples/vhost/main.c +1 -1
  	if (async_vhost_driver) {
  		if (!enable) {
  			uint16_t n_pkt = 0;
-			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+			pkts_inflight =

rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);

examples/vhost/main.c destroy_device(int vid)

Quoted 5 lines
examples/vhost/main.c +2 -1
+			struct rte_mbuf *m_cpl[pkts_inflight];
 
-			while (vdev->pkts_inflight) {
+			while (pkts_inflight) {
  				n_pkt =

rte_vhost_clear_queue_thread_unsafe(vid, queue_id,

Quoted 8 lines

examples/vhost/main.c vring_state_changed(int vid, uint16_t queue_id, · line 1510

Quoted 5 lines
examples/vhost/main.c +1 -2
-							m_cpl, vdev-
 pkts_inflight);
+							m_cpl, pkts_inflight);
  				free_pkts(m_cpl, n_pkt);
-				__atomic_sub_fetch(&vdev->pkts_inflight,

n_pkt, __ATOMIC_SEQ_CST);

Same comments as destroy_device

Same as vring_state_changed.

Regards,
Xuan

Quoted 17 lines
Quoted 17 lines
examples/vhost/main.c +0 -0
  			}
  		}
  	}
examples/vhost/main.h +0 -1
index e7b1ac60a6..0ccdce4b4a 100644
@@ -51,7 +51,6 @@ struct vhost_dev {
  	uint64_t features;
  	size_t hdr_len;
  	uint16_t nr_vrings;
-	uint16_t pkts_inflight;
  	struct rte_vhost_memory *mem;
  	struct device_statistics stats;
  	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
author
#16 lore raw

v6 cover add unsafe API to get inflight packets

In reply to Xuan Ding

This patchset introduces an unsafe API to get the number of inflight packets in DMA engine in some situations. Like vring state changes or device is destroyed. Compared with rte_vhost_async_get_inflight(), this is a lock free version.

v6:
* Fixed an issue of infinite loop.

v5:
* Rebased to the lastest branch.

v4:
* Added use case for API in vhost example.

v3:
* Fixed one typo.
* Revised the doc to be more accuracy.

v2:
* Fixed some format issues.

Xuan Ding (2):
vhost: add unsafe API to check inflight packets examples/vhost: use API to check inflight packets

 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 examples/vhost/main.c                  | 31 ++++++++++++++------------
 examples/vhost/main.h                  |  1 -
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 +++++++++++++++++++++
 7 files changed, 70 insertions(+), 15 deletions(-)
Signature · 1 line
2.17.1
author
#17 lore raw

PATCH v6 1/2 vhost: add unsafe API to check inflight packets

In async data path, when vring state changes or device is destroyed, it is necessary to know the number of inflight packets in DMA engine. This patch provides a thread unsafe API to return the number of inflight packets for a vhost queue without using any lock.

---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)
doc/guides/prog_guide/vhost_lib.rst +5 -0
index 171e0096f6..73a4e24640 100644
@@ -298,6 +298,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+* ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
doc/guides/rel_notes/release_21_11.rst +5 -0
index ad7c1afec0..fd821b29d3 100644
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path without using lock.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
lib/vhost/rte_vhost_async.h +14 -0
index ad71555a7f..1f0708378b 100644
@@ -234,6 +234,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
lib/vhost/version.map +3 -0
index 8ebde3f694..ed6cb53a99 100644
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
lib/vhost/vhost.c +26 -0
index 996287c3c2..c16c6bc2bb 100644
@@ -1884,5 +1884,31 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int
+rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
Signature · 1 line
2.17.1
author
#18 lore raw

PATCH v6 2/2 examples/vhost: use API to check inflight packets

In reply to Xuan Ding

In async data path, call rte_vhost_async_get_inflight_thread_unsafe() API to directly return the number of inflight packets instead of maintaining a local variable.

---
 examples/vhost/main.c | 31 +++++++++++++++++--------------
 examples/vhost/main.h |  1 -
 2 files changed, 17 insertions(+), 15 deletions(-)
examples/vhost/main.c +17 -14
index d0bf1f31e3..7a5d97cef6 100644
@@ -842,11 +842,8 @@ complete_async_pkts(struct vhost_dev *vdev)
 
 	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
 					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
-	if (complete_count) {
+	if (complete_count)
 		free_pkts(p_cpl, complete_count);
-		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count, __ATOMIC_SEQ_CST);
-	}
-
 }
 
 static __rte_always_inline void
@@ -886,7 +883,6 @@ drain_vhost(struct vhost_dev *vdev)
 
 		complete_async_pkts(vdev);
 		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ, m, nr_xmit);
-		__atomic_add_fetch(&vdev->pkts_inflight, ret, __ATOMIC_SEQ_CST);
 
 		enqueue_fail = nr_xmit - ret;
 		if (enqueue_fail)
@@ -1212,7 +1208,6 @@ drain_eth_rx(struct vhost_dev *vdev)
 		complete_async_pkts(vdev);
 		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
 					VIRTIO_RXQ, pkts, rx_count);
-		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count, __ATOMIC_SEQ_CST);
 
 		enqueue_fail = rx_count - enqueue_count;
 		if (enqueue_fail)
@@ -1384,13 +1379,17 @@ destroy_device(int vid)
 
 	if (async_vhost_driver) {
 		uint16_t n_pkt = 0;
-		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+		int pkts_inflight;
 
-		while (vdev->pkts_inflight) {
+		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, VIRTIO_RXQ);
+		struct rte_mbuf *m_cpl[pkts_inflight];
+
+		while (pkts_inflight) {
 			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, VIRTIO_RXQ,
-						m_cpl, vdev->pkts_inflight);
+						m_cpl, pkts_inflight);
 			free_pkts(m_cpl, n_pkt);
-			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
+			pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid,
+									VIRTIO_RXQ);
 		}
 
 		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
@@ -1500,13 +1499,17 @@ vring_state_changed(int vid, uint16_t queue_id, int enable)
 	if (async_vhost_driver) {
 		if (!enable) {
 			uint16_t n_pkt = 0;
-			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+			int pkts_inflight;
+
+			pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);
+			struct rte_mbuf *m_cpl[pkts_inflight];
 
-			while (vdev->pkts_inflight) {
+			while (pkts_inflight) {
 				n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, queue_id,
-							m_cpl, vdev->pkts_inflight);
+							m_cpl, pkts_inflight);
 				free_pkts(m_cpl, n_pkt);
-				__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
+				pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid,
+										queue_id);
 			}
 		}
 	}
examples/vhost/main.h +0 -1
index e7b1ac60a6..0ccdce4b4a 100644
@@ -51,7 +51,6 @@ struct vhost_dev {
 	uint64_t features;
 	size_t hdr_len;
 	uint16_t nr_vrings;
-	uint16_t pkts_inflight;
 	struct rte_vhost_memory *mem;
 	struct device_statistics stats;
 	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
Signature · 1 line
2.17.1