doc: clarify virtio PMD path selection
PATCH 1/1 doc: clarify virtio PMD path selection
From: Yinan Wang <[email protected]>
add virtio paths selection and usage introduction for better virtio usability.
--- .../virtio_paths_selection_and_usage.rst | 130 ++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rst
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +130 -0
@@ -0,0 +1,130 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Virtio paths Selection and Usage +================================ + +Logically virtio-PMD has 9 paths based on the virtio features (Rx mergeable,In-order,Packed virtqueue) +combinations, below are introduction of virtio three common features: + +* Rx mergeable: With this feature negotiated, device can receive larger packets by combining + individual descriptors. +* In-order: Some devices always use descriptors in the same order in which they have been made + available, these devices can offer the VIRTIO_F_IN_ORDER feature. If this feature negotiated, + driver will use descriptors in order. Meanwhile, this knowledge allows device operate used ring + in batches and driver operate available ring in batches and such can decrease cache miss rate. +* Packed virtqueue: The structure of packed virtqueue is different from split virtqueue,split + virtqueue is composed of available ring, used ring and descriptor table, while packed virtqueue + is composed of descriptor ring,driver event suppression and device event suppression. The idea + behind this is to improve performance by avoiding cache misses and and make it easier for devices + to implement. + +Virtio paths Selection +---------------------- + +If packed virtqueue is not negotiated, below split virtqueue paths can be selected +according to below configuration: + +#. Split virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature is + not negotiated, this path will be selected. +#. Split virtqueue non-mergeable path: If Rx mergeable and in-order feature are not + negotiated, also Rx offload(s) are requested, this path can be selected. +#. Split virtqueue in-order mergeable path: If in-order feature and Rx mergeable are + both negotiated, this path can be selected. +#. Split virtqueue in-order non-mergeable path: If in-order feature is negotiated and + Rx mergeable is not negotiated, this path can be selected. +#. Split virtqueue vectorized RX path: If Rx mergeable is disabled and no Rx offload + requested, this path can be selected. + +If packed virtqueue is negotiated, below packed virtqueue paths can be selected +according to below configuration: + +#. Packed virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature + is not negotiated, this path will be selected. +#. Packed virtqueue non-mergeable path: If Rx mergeable and in-order feature are not + negotiated, also Rx offload(s) are requested, this path will be selected. +#. Packed virtqueue in-order mergeable path: If in-order feature and Rx mergeable are + both negotiated, this path will be selected. +#. Packed virtqueue in-order non-mergeable path: If in-order feature is negotiated and + Rx mergeable is not negotiated, this path will be selected. + +Rx/Tx callbacks of each Virtio path +----------------------------------- + +Refer to above descriptions,virtio path and Rx/TX callbacks are auto selected by different parameters of +vdev and workloads. Rx callbacks and Tx callbacks name for each Virtio Path are shown in following tables:: + + +----------------------------------------------------------------------------------------------------------+ + | Virtio path | Rx callbacks | TX callbacks | + +----------------------------------------------------------------------------------------------------------+ + |Split virtqueue mergeable path |virtio_recv_mergeable_pkts | virtio_xmit_pkts | + +----------------------------------------------------------------------------------------------------------+ + |Split virtqueue non-mergeable path | virtio_recv_pkts | virtio_xmit_pkts | + +----------------------------------------------------------------------------------------------------------+ + |Split virtqueue in-order mergeable path | virtio_recv_pkts_inorder | virtio_xmit_pkts_inorder| + +----------------------------------------------------------------------------------------------------------+ + |Split virtqueue in-order non-mergeable path | virtio_recv_pkts_inorder | virtio_xmit_pkts_inorder| + +----------------------------------------------------------------------------------------------------------+ + |Split virtqueue vectorized RX path | virtio_recv_pkts_vec | virtio_xmit_pkts | + +----------------------------------------------------------------------------------------------------------+ + |Packed virtqueue mergeable path | virtio_recv_mergeable_pkts_packed| virtio_xmit_pkts_packed | + +----------------------------------------------------------------------------------------------------------+ + |Packed virtqueue normal path | virtio_recv_pkts_packed | virtio_xmit_pkts_packed | + +----------------------------------------------------------------------------------------------------------+ + |Packed virtqueue in-order mergeable path | virtio_recv_mergeable_pkts_packed| virtio_xmit_pkts_packed | + +----------------------------------------------------------------------------------------------------------+ + |Packed virtqueue in-order normal path | virtio_recv_pkts_packed | virtio_xmit_pkts_packed | + +----------------------------------------------------------------------------------------------------------+ + +Virtio paths Support Status from Release to Release +--------------------------------------------------- + +Virtio feature implementation: + +* In-order feature implemented in DPDK 18.08 by adding new Rx/TX callbacks + ``virtio_recv_pkts_inorder`` and ``virtio_xmit_pkts_inorder``. +* Packed virtqueue implemented in DPDK 19.02 by adding new Rx/TX callbacks + ``virtio_recv_pkts_packed`` , ``virtio_recv_mergeable_pkts_packed`` and ``virtio_xmit_pkts_packed``. + +Virtio path number changes from release to release, all virtio paths support status are shown in below table:: + + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Virtio path\ DPDK version | v16.11 | v17.02 | v17.05 | v17.08 | v17.11 | v18.02 | v18.05 | v18.08 | v18.11 | v19.02 | v19.05 | v19.08 | + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue mergebale path | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue non-mergeable path | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue vectorized RX path | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue simple TX path | Y | Y | Y | Y | Y | Y | Y | N | N | N | N | N | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue in-order non-mergeable path | | | | | | | | Y | Y | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue in-order mergeable path | | | | | | | | Y | Y | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Packed virtqueue mergeable path | | | | | | | | | | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Packed virtqueue non-mergeable path | | | | | | | | | | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Packed virtqueue in-order mergeable path | | | | | | | | | | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Packed virtqueue in-order non-mergeable path| | | | | | | | | | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + +QEMU Support Status +------------------- + +Qemu only support three path of Virtio-PMD: Split virtqueue mergebale path, +Split virtqueue no-mergeable path,Split virtqueue vectorized RX path. + +How to Debug +------------ + +If you meet performance drop or some other issues after upgrading the driver +or configuration, below steps can help you identify which path you selected and +root cause faster. + +#. Run vhost/virtio test case; +#. Run "perf top" and check virtio Rx/tx callback names; +#. Identify which virtio path is selected refer to above table.
Signature · 1 line
2.17.1
Quoted 7 lines · On Mon, 18 Nov 2019 05:29:23 -0500 Yinan <[email protected]> wrote:
From: Yinan Wang <[email protected]>
add virtio paths selection and usage introduction for better virtio usability.
Signed-off-by Yinan Wang <[email protected]>---
Nice to have this documented.
Do you think it would make sense to link to the virtio standard that defines this?
Thanks your suggestion, I will add the link in v2 version.
Quoted 24 lines
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: 2019年11月18日 10:24
To: Wang, Yinan <[email protected]>
Cc: [email protected]; [email protected]; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>; Mcnamara, John <[email protected]>; Kovacevic, Marko <[email protected]>
Subject: Re: [dpdk-dev] [PATCH] doc: clarify virtio PMD path selectionQuoted 7 lines · On Mon, 18 Nov 2019 05:29:23 -0500 Yinan <[email protected]> wrote:
From: Yinan Wang <[email protected]>
add virtio paths selection and usage introduction for better virtio usability.
Signed-off-by Yinan Wang <[email protected]>---
Nice to have this documented.
Do you think it would make sense to link to the virtio standard that defines this?
Hi Yinan,
Quoted 33 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Yinan
Sent: Monday, November 18, 2019 6:29 PM
To: [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Yinan Wang <[email protected]>
Subject: [dpdk-dev] [PATCH] doc: clarify virtio PMD path selectionFrom: Yinan Wang <[email protected]>
add virtio paths selection and usage introduction for better virtio usability.
Signed-off-by Yinan Wang <[email protected]>--- .../virtio_paths_selection_and_usage.rst | 130 ++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rstdoc/guides/howto/virtio_paths_selection_and_usage.rst new file +7
-0@@ -0,0 +1,130 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Virtio paths Selection and Usage +================================ + +Logically virtio-PMD has 9 paths based on the virtio features (Rx
Quoted 2 lines
mergeable,In-order,Packed virtqueue)
+combinations, below are introduction of virtio three common features:
s/are/is an
Quoted 172 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ +* Rx mergeable: With this feature negotiated, device can receive largerpackets by combining
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ individual descriptors. +* In-order: Some devices always use descriptors in the same order in whichthey have been made
+ available, these devices can offer theVIRTIO_F_IN_ORDERfeature. If this
feature negotiated,
+ driver will use descriptors in order. Meanwhile, this knowledge allows
device operate used ring
+ in batches and driver operate available ring in batches and such can
decrease cache miss rate.
+* Packed virtqueue: The structure of packed virtqueue is different from split
virtqueue,split
+ virtqueue is composed of available ring, used ring and descriptor table,
while packed virtqueue
+ is composed of descriptor ring,driver event suppression and device event
suppression. The idea
+ behind this is to improve performance by avoiding cache misses and and
make it easier for devicesdoc/guides/howto/virtio_paths_selection_and_usage.rst new file +6
-0+ to implement. + +Virtio paths Selection +---------------------- + +If packed virtqueue is not negotiated, below split virtqueue paths can beselected
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +3
-0+according to below configuration: + +#. Split virtqueue mergeable path: If Rx mergeable is negotiated, in-orderfeature is
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ not negotiated, this path will be selected. +#. Split virtqueue non-mergeable path: If Rx mergeable and in-order featureare not
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ negotiated, also Rx offload(s) are requested, this path can be selected. +#. Split virtqueue in-order mergeable path: If in-order feature and Rxmergeable are
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ both negotiated, this path can be selected. +#. Split virtqueue in-order non-mergeable path: If in-order feature isnegotiated and
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ Rx mergeable is not negotiated, this path can be selected. +#. Split virtqueue vectorized RX path: If Rx mergeable is disabled and no Rxoffload
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +3
-0+ requested, this path can be selected. + +If packed virtqueue is negotiated, below packed virtqueue paths can beselected
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +3
-0+according to below configuration: + +#. Packed virtqueue mergeable path: If Rx mergeable is negotiated, in-orderfeature
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ is not negotiated, this path will be selected. +#. Packed virtqueue non-mergeable path: If Rx mergeable and in-orderfeature are not
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ negotiated, also Rx offload(s) are requested, this path will be selected. +#. Packed virtqueue in-order mergeable path: If in-order feature and Rxmergeable are
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ both negotiated, this path will be selected. +#. Packed virtqueue in-order non-mergeable path: If in-order feature isnegotiated and
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +6
-0+ Rx mergeable is not negotiated, this path will be selected. + +Rx/Tx callbacks of each Virtio path +----------------------------------- + +Refer to above descriptions,virtio path and Rx/TX callbacks are auto selectedby different parameters of
+vdev and workloads. Rx callbacks and Tx callbacks name for each Virtio Path are shown in following tables::doc/guides/howto/virtio_paths_selection_and_usage.rst new file +5
-2+ + +----------------------------------------------------------------------------------------------- -----------+ + | Virtio path | Rx callbacks | TX callbacks | + +----------------------------------------------------------------------------------------------- -----------+ + |Split virtqueue mergeable path |virtio_recv_mergeable_pkts |
virtio_xmit_pkts|doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- -----------+ + |Split virtqueue non-mergeable path | virtio_recv_pkts |
virtio_xmit_pkts|doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- -----------+ + |Split virtqueue in-order mergeable path | virtio_recv_pkts_inorder |virtio_xmit_pkts_inorder|
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- -----------+ + |Split virtqueue in-order non-mergeable path | virtio_recv_pkts_inorder| virtio_xmit_pkts_inorder|
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- -----------+ + |Split virtqueue vectorized RX path | virtio_recv_pkts_vec |
virtio_xmit_pkts|doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- -----------+ + |Packed virtqueue mergeable path |virtio_recv_mergeable_pkts_packed|
virtio_xmit_pkts_packed|doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- -----------+ + |Packed virtqueue normal path | virtio_recv_pkts_packed |
virtio_xmit_pkts_packed|doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- -----------+ + |Packed virtqueue in-order mergeable path |virtio_recv_mergeable_pkts_packed|
virtio_xmit_pkts_packed|doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- -----------+ + |Packed virtqueue in-order normal path | virtio_recv_pkts_packed|
virtio_xmit_pkts_packed|doc/guides/howto/virtio_paths_selection_and_usage.rst new file +8
-1+ +----------------------------------------------------------------------------------------------- -----------+ + +Virtio paths Support Status from Release to Release +--------------------------------------------------- + +Virtio feature implementation: + +* In-order feature implemented in DPDK 18.08 by adding new Rx/TXcallbacks
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ ``virtio_recv_pkts_inorder`` and ``virtio_xmit_pkts_inorder``. +* Packed virtqueue implemented in DPDK 19.02 by adding new Rx/TXcallbacks
+ ``virtio_recv_pkts_packed`` , ``virtio_recv_mergeable_pkts_packed`` and
``virtio_xmit_pkts_packed``.doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ +Virtio path number changes from release to release, all virtio paths supportstatus are shown in below table::
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +3
-1+ + +----------------------------------------------------------------------------------------------- ---------------------------------------------------------+ + |Virtio path\ DPDK version | v16.11 | v17.02 | v17.05 | v17.08 |v17.11 | v18.02 | v18.05 | v18.08 | v18.11 | v19.02 | v19.05 | v19.08 |
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- ---------------------------------------------------------+ + |Split virtqueue mergebale path | Y | Y | Y | Y | Y | Y| Y | Y | Y | Y | Y | Y |
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- ---------------------------------------------------------+ + |Split virtqueue non-mergeable path | Y | Y | Y | Y | Y | Y| Y | Y | Y | Y | Y | Y |
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ +----------------------------------------------------------------------------------------------- ---------------------------------------------------------+ + |Split virtqueue vectorized RX path | Y | Y | Y | Y | Y | Y| Y | Y | Y | Y | Y | Y |
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ ------------------------------------------------------------------------------------------------- --------------------------------------------------------+ + |Split virtqueue simple TX path | Y | Y | Y | Y | Y | Y |Y | N | N | N | N | N |
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ ------------------------------------------------------------------------------------------------- --------------------------------------------------------+ + |Split virtqueue in-order non-mergeable path | | | | | || | Y | Y | Y | Y | Y |
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ ------------------------------------------------------------------------------------------------- --------------------------------------------------------+ + |Split virtqueue in-order mergeable path | | | | | || | Y | Y | Y | Y | Y |
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ ------------------------------------------------------------------------------------------------- --------------------------------------------------------+ + |Packed virtqueue mergeable path | | | | | | || | | Y | Y | Y |
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ ------------------------------------------------------------------------------------------------- --------------------------------------------------------+ + |Packed virtqueue non-mergeable path | | | | | || | | | Y | Y | Y |
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ ------------------------------------------------------------------------------------------------- --------------------------------------------------------+ + |Packed virtqueue in-order mergeable path | | | | | || | | | Y | Y | Y |
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-1+ ------------------------------------------------------------------------------------------------- --------------------------------------------------------+ + |Packed virtqueue in-order non-mergeable path| | | | || | | | | Y | Y | Y |
doc/guides/howto/virtio_paths_selection_and_usage.rst line 116
Quoted 5 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +4
-1+ ------------------------------------------------------------------------------------------------- --------------------------------------------------------+ + +QEMU Support Status +-------------------
It is better to add Qemu version latest, the future Qemu version may support more.
doc/guides/howto/virtio_paths_selection_and_usage.rst
Quoted 2 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ +Qemu only support three path of Virtio-PMD: Split virtqueue mergebale path,
Add "s" to the first path.
Quoted 14 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +7
-0+Split virtqueue no-mergeable path,Split virtqueue vectorized RX path. + +How to Debug +------------ + +If you meet performance drop or some other issues after upgrading the driver +or configuration, below steps can help you identify which path you selectedand
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +5
-1+root cause faster. + +#. Run vhost/virtio test case; +#. Run "perf top" and check virtio Rx/tx callback names; +#. Identify which virtio path is selected refer to above table. --
Quoted 1 line
2.17.1
Other than the minor comments,
PATCH v4 1/1 doc: clarify virtio PMD path selection
In reply to YinanFrom: Wang Yinan <[email protected]>
add virtio paths selection and usage introduction for better virtio usability.
--- doc/guides/howto/index.rst | 1 + .../virtio_paths_selection_and_usage.rst | 142 ++++++++++++++++++ 2 files changed, 143 insertions(+) create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rst
doc/guides/howto/index.rst +1 -0
@@ -16,6 +16,7 @@ HowTo Guides vfd virtio_user_for_container_networking virtio_user_as_exceptional_path + virtio_paths_selection_and_usage packet_capture_framework telemetry debug_troubleshoot
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +142 -0
142 lines not on this page · raw message
Signature · 1 line
2.17.1
Quoted 3 lines · Yinan
From: Wang Yinan <[email protected]>
add virtio paths selection and usage introduction for better
s/add/Add/
Quoted 27 lines
virtio usability.
Signed-off-by Wang Yinan <[email protected]>--- doc/guides/howto/index.rst | 1 + .../virtio_paths_selection_and_usage.rst | 142 ++++++++++++++++++ 2 files changed, 143 insertions(+) create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rstdoc/guides/howto/index.rst +1
-0@@ -16,6 +16,7 @@ HowTo Guides vfd virtio_user_for_container_networking virtio_user_as_exceptional_path + virtio_paths_selection_and_usage packet_capture_framework telemetry debug_troubleshootdoc/guides/howto/virtio_paths_selection_and_usage.rst new file +0
-0
doc/guides/howto/virtio_paths_selection_and_usage.rst line 9
Quoted 9 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +9
-0@@ -0,0 +1,142 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Intel Corporation. + +Virtio paths Selection and Usage +================================ + +Logically virtio-PMD has 9 paths based on the combination of virtio features +(Rx mergeable, In-order, Packed virtqueue), below is an introduction of virtio +common features:
s/virtio common features/these features/
Quoted 7 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +7
-0+ +* `Rx mergeable <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/ + virtio-v1.1-cs01.html#x1-2140004>`_: With this feature negotiated, device + can receive larger packets by combining individual descriptors. +* `In-order <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/ + virtio-v1.1-cs01.html#x1-690008>`_: Some devices always use descriptors + in the same order in which they have been made available, these
doc/guides/howto/virtio_paths_selection_and_usage.rst line 26
Quoted 10 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +10
-0+ devices can offer the VIRTIO_F_IN_ORDER feature. If this feature negotiated, + driver will use descriptors in order. Meanwhile, this knowledge allows device + operate used ring in batches and driver operate available ring in batches and + such can decrease cache miss rate. +* `Packed virtqueue <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/ + virtio-v1.1-cs01.html#x1-610007>`_: The structure of packed virtqueue is + different from split virtqueue, split virtqueue is composed of available ring, + used ring and descriptor table, while packed virtqueue is composed of descriptor + ring, driver event suppression and device event suppression. The idea behind + this is to improve performance by avoiding cache misses and and make it easier
s/and and/and/
Quoted 7 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +7
-0+ for devices to implement. + +Virtio paths Selection +---------------------- + +If packed virtqueue is not negotiated, below split virtqueue paths will be selected +according to below configuration:
doc/guides/howto/virtio_paths_selection_and_usage.rst line 43
Quoted 10 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +10
-0+ +#. Split virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature is + not negotiated, this path will be selected. +#. Split virtqueue non-mergeable path: If Rx mergeable and in-order feature are not + negotiated, also Rx offload(s) are requested, this path will be selected. +#. Split virtqueue in-order mergeable path: If Rx mergeable and in-order feature are + both negotiated, this path will be selected. +#. Split virtqueue in-order non-mergeable path: If in-order feature is negotiated and + Rx mergeable is not negotiated, this path will be selected. +#. Split virtqueue vectorized RX path: If Rx mergeable is disabled and no Rx offload
s/RX/Rx/
Quoted 8 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +8
-0+ requested, this path will be selected. + +If packed virtqueue is negotiated, below packed virtqueue paths will be selected +according to below configuration: + +#. Packed virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature + is not negotiated, this path will be selected. +#. Packed virtqueue non-mergeable path: If Rx mergeable and in-order feature are not
doc/guides/howto/virtio_paths_selection_and_usage.rst line 61
Quoted 10 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +10
-0+ negotiated, this path will be selected. +#. Packed virtqueue in-order mergeable path: If in-order and Rx mergeable feature are + both negotiated, this path will be selected. +#. Packed virtqueue in-order non-mergeable path: If in-order feature is negotiated and + Rx mergeable is not negotiated, this path will be selected. + +Rx/Tx callbacks of each Virtio path +----------------------------------- + +Refer to above descriptions, virtio path and Rx/TX callbacks are auto selected by
s/TX/Tx/
doc/guides/howto/virtio_paths_selection_and_usage.rst line 63
Quoted 2 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+different parameters of vdev and workloads. Rx callbacks and Tx callbacks name for +each Virtio Path are shown in following tables::
s/in following tables::/in below table:/
doc/guides/howto/virtio_paths_selection_and_usage.rst line 66
Quoted 3 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +3
-0+ + +----------------------------------------------------------------------------------------------------------+ + | Virtio path | Rx callbacks | TX callbacks |
s/TX/Tx/
doc/guides/howto/virtio_paths_selection_and_usage.rst line 76
Quoted 10 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +10
-0+ +----------------------------------------------------------------------------------------------------------+ + |Split virtqueue mergeable path |virtio_recv_mergeable_pkts | virtio_xmit_pkts | + +----------------------------------------------------------------------------------------------------------+ + |Split virtqueue non-mergeable path | virtio_recv_pkts | virtio_xmit_pkts | + +----------------------------------------------------------------------------------------------------------+ + |Split virtqueue in-order mergeable path | virtio_recv_pkts_inorder | virtio_xmit_pkts_inorder| + +----------------------------------------------------------------------------------------------------------+ + |Split virtqueue in-order non-mergeable path | virtio_recv_pkts_inorder | virtio_xmit_pkts_inorder| + +----------------------------------------------------------------------------------------------------------+ + |Split virtqueue vectorized RX path | virtio_recv_pkts_vec | virtio_xmit_pkts |
s/RX/Rx/
doc/guides/howto/virtio_paths_selection_and_usage.rst line 80
Quoted 4 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +4
-0+ +----------------------------------------------------------------------------------------------------------+ + |Packed virtqueue mergeable path | virtio_recv_mergeable_pkts_packed| virtio_xmit_pkts_packed | + +----------------------------------------------------------------------------------------------------------+ + |Packed virtqueue normal path | virtio_recv_pkts_packed | virtio_xmit_pkts_packed |
s/normal/non-mergeable/
doc/guides/howto/virtio_paths_selection_and_usage.rst line 84
Quoted 4 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +4
-0+ +----------------------------------------------------------------------------------------------------------+ + |Packed virtqueue in-order mergeable path | virtio_recv_mergeable_pkts_packed| virtio_xmit_pkts_packed | + +----------------------------------------------------------------------------------------------------------+ + |Packed virtqueue in-order normal path | virtio_recv_pkts_packed | virtio_xmit_pkts_packed |
s/normal/non-mergeable/
doc/guides/howto/virtio_paths_selection_and_usage.rst line 85
Quoted 1 line
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +1
-0+ +----------------------------------------------------------------------------------------------------------+
It seems above table will be interpreted as code block. You can use below table:
.. table:: Virtio Paths and Callbacks
============================================ ================================= ========================
Virtio paths Rx callbacks Tx callbacks
============================================ ================================= ========================
Split virtqueue mergeable path virtio_recv_mergeable_pkts virtio_xmit_pkts Split virtqueue non-mergeable path virtio_recv_pkts virtio_xmit_pkts Split virtqueue in-order mergeable path virtio_recv_pkts_inorder virtio_xmit_pkts_inorder Split virtqueue in-order non-mergeable path virtio_recv_pkts_inorder virtio_xmit_pkts_inorder Split virtqueue vectorized Rx path virtio_recv_pkts_vec virtio_xmit_pkts Packed virtqueue mergeable path virtio_recv_mergeable_pkts_packed virtio_xmit_pkts_packed Packed virtqueue non-meregable path virtio_recv_pkts_packed virtio_xmit_pkts_packed Packed virtqueue in-order mergeable path virtio_recv_mergeable_pkts_packed virtio_xmit_pkts_packed
Packed virtqueue in-order non-mergeable path virtio_recv_pkts_packed virtio_xmit_pkts_packed
============================================ ================================= ========================
doc/guides/howto/virtio_paths_selection_and_usage.rst line 92
Quoted 7 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +7
-0+ +Virtio paths Support Status from Release to Release +--------------------------------------------------- + +Virtio feature implementation: + +* In-order feature implemented in DPDK 18.08 by adding new Rx/TX callbacks
s/TX/Tx/
s/implemented in/is supported since/
doc/guides/howto/virtio_paths_selection_and_usage.rst line 94
Quoted 2 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2
-0+ ``virtio_recv_pkts_inorder`` and ``virtio_xmit_pkts_inorder``. +* Packed virtqueue implemented in DPDK 19.02 by adding new Rx/TX callbacks
Ditto.
Quoted 19 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +19
-0+ ``virtio_recv_pkts_packed`` , ``virtio_recv_mergeable_pkts_packed`` and + ``virtio_xmit_pkts_packed``. + +Virtio path number changes from release to release, all virtio paths support +status are shown in below table:: + + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Virtio path\ DPDK version | v16.11 | v17.02 | v17.05 | v17.08 | v17.11 | v18.02 | v18.05 | v18.08 | v18.11 | v19.02 | v19.05 | v19.08 | + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue mergeable path | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue non-mergeable path | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue vectorized RX path | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue simple TX path | Y | Y | Y | Y | Y | Y | Y | N | N | N | N | N | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Split virtqueue in-order non-mergeable path | | | | | | | | Y | Y | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+
doc/guides/howto/virtio_paths_selection_and_usage.rst line 123
Quoted 10 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +10
-0+ |Split virtqueue in-order mergeable path | | | | | | | | Y | Y | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Packed virtqueue mergeable path | | | | | | | | | | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Packed virtqueue non-mergeable path | | | | | | | | | | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Packed virtqueue in-order mergeable path | | | | | | | | | | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ + |Packed virtqueue in-order non-mergeable path| | | | | | | | | | Y | Y | Y | + ---------------------------------------------------------------------------------------------------------------------------------------------------------+
19.11 can be included in above table as well.
Columns can be merged if possible.
You can do it like this:
.. table:: Virtio Paths and Releases
============================================ ============= ============= =============
Virtio paths 16.11 ~ 18.05 18.08 ~ 18.11 19.02 ~ 19.11 ============================================ ============= ============= ============= Split virtqueue mergeable path Y Y Y Split virtqueue non-mergeable path Y Y Y Split virtqueue vectorized Rx path Y Y Y Split virtqueue simple Tx path Y N N Split virtqueue in-order mergeable path Y Y Split virtqueue in-order non-mergeable path Y Y Packed virtqueue mergeable path Y Packed virtqueue non-mergeable path Y Packed virtqueue in-order mergeable path Y Packed virtqueue in-order non-mergeable path Y
============================================ ============= ============= =============
doc/guides/howto/virtio_paths_selection_and_usage.rst line 129
Quoted 6 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +6
-0+ +QEMU Support Status +------------------- + +* Qemu now supports three paths of split virtqueue: Split virtqueue mergeable path, + Split virtqueue non-mergeable path, Split virtqueue vectorized RX path.
s/RX/Rx/
Quoted 16 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +13
-0+* Since qemu 4.2.0, Packed virtqueue mergeable path and Packed virtqueue non-mergeable + path can be supported. + +How to Debug +------------ + +If you meet performance drop or some other issues after upgrading the driver +or configuration, below steps can help you identify which path you selected and +root cause faster. + +#. Run vhost/virtio test case; +#. Run "perf top" and check virtio Rx/tx callback names; +#. Identify which virtio path is selected refer to above table.Signature · 1 line
2.17.1
18/11/2019 11:29, Yinan:
+--------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Virtio path\ DPDK version | v16.11 | v17.02 | v17.05 | v17.08 | v17.11 | v18.02 | v18.05 | v18.08 | v18.11 | v19.02 | v19.05 | v19.08 | > + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Split virtqueue mergebale path | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | > + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Split virtqueue non-mergeable path | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | > + +--------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Split virtqueue vectorized RX path | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | > + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Split virtqueue simple TX path | Y | Y | Y | Y | Y | Y | Y | N | N | N | N | N | > + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Split virtqueue in-order non-mergeable path | | | | | | | | Y | Y | Y | Y | Y | > + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Split virtqueue in-order mergeable path | | | | | | | | Y | Y | Y | Y | Y | > + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Packed virtqueue mergeable path | | | | | | | | | | Y | Y | Y | > + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Packed virtqueue non-mergeable path | | | | | | | | | | Y | Y | Y | > + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Packed virtqueue in-order mergeable path | | | | | | | | | | Y | Y | Y | > + ---------------------------------------------------------------------------------------------------------------------------------------------------------+ > + |Packed virtqueue in-order non-mergeable path| | | | | | | | | | Y | Y | Y | > + ---------------------------------------------------------------------------------------------------------------------------------------------------------+
Please try to make the table shorter.
I think there would be some gain to name versions without "v" and use a simple table syntax:
http://docutils.sourceforge.net/docs/user/rst/quickref.html#tables
PATCH v5 1/1 doc: clarify virtio PMD path selection
In reply to YinanFrom: Yinan Wang <[email protected]>
Add virtio paths selection and usage introduction for better virtio usability.
--- doc/guides/howto/index.rst | 1 + .../virtio_paths_selection_and_usage.rst | 126 ++++++++++++++++++ 2 files changed, 127 insertions(+) create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rst
doc/guides/howto/index.rst +1 -0
@@ -16,6 +16,7 @@ HowTo Guides vfd virtio_user_for_container_networking virtio_user_as_exceptional_path + virtio_paths_selection_and_usage packet_capture_framework telemetry debug_troubleshoot
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +126 -0
126 lines not on this page · raw message
Signature · 1 line
2.17.1
Quoted 11 lines · Yinan
From: Yinan Wang <[email protected]>
Add virtio paths selection and usage introduction for better virtio usability.
Signed-off-by Yinan Wang <[email protected]>--- doc/guides/howto/index.rst | 1 + .../virtio_paths_selection_and_usage.rst | 126 ++++++++++++++++++ 2 files changed, 127 insertions(+) create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rst
Quoted 5 lines · Yinan
From: Yinan Wang <[email protected]>
Add virtio paths selection and usage introduction for better virtio usability.
Quoted 7 lines
Signed-off-by Yinan Wang <[email protected]>--- doc/guides/howto/index.rst | 1 + .../virtio_paths_selection_and_usage.rst | 126 ++++++++++++++++++ 2 files changed, 127 insertions(+) create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rst
Thanks,
Maxime
27/11/2019 16:19, Yinan:
Quoted 9 lines
From: Yinan Wang <[email protected]>
Add virtio paths selection and usage introduction for better virtio usability.
Signed-off-by Yinan Wang <[email protected]>--- doc/guides/howto/index.rst | 1 + .../virtio_paths_selection_and_usage.rst | 126 ++++++++++++++++++
I don't understand why this doc is added in howto guides, and not in doc/guides/nics/virtio.rst ?
PATCH v6 1/1 doc: clarify virtio PMD path selection
In reply to YinanFrom: Yinan Wang <[email protected]>
Add virtio paths selection and usage introduction for better virtio usability.
---
v6:
- Move to Virtio NIC documentation
doc/guides/nics/virtio.rst | 124 +++++++++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+)
doc/guides/nics/virtio.rst +124 -0
@@ -402,3 +402,127 @@ Below devargs are supported by the virtio-user vdev: It is used to enable virtio device packed virtqueue feature. (Default: 0 (disabled)) + +Virtio paths Selection and Usage +-------------------------------- + +Logically virtio-PMD has 9 paths based on the combination of virtio features +(Rx mergeable, In-order, Packed virtqueue), below is an introduction of these +features: + +* `Rx mergeable <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/ + virtio-v1.1-cs01.html#x1-2140004>`_: With this feature negotiated, device + can receive large packets by combining individual descriptors. +* `In-order <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/ + virtio-v1.1-cs01.html#x1-690008>`_: Some devices always use descriptors + in the same order in which they have been made available, these + devices can offer the VIRTIO_F_IN_ORDER feature. With this feature negotiated, + driver will use descriptors in order. +* `Packed virtqueue <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/ + virtio-v1.1-cs01.html#x1-610007>`_: The structure of packed virtqueue is + different from split virtqueue, split virtqueue is composed of available ring, + used ring and descriptor table, while packed virtqueue is composed of descriptor + ring, driver event suppression and device event suppression. The idea behind + this is to improve performance by avoiding cache misses and make it easier + for hardware to implement. + +Virtio paths Selection +~~~~~~~~~~~~~~~~~~~~~~ + +If packed virtqueue is not negotiated, below split virtqueue paths will be selected +according to below configuration: + +#. Split virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature is + not negotiated, this path will be selected. +#. Split virtqueue non-mergeable path: If Rx mergeable and in-order feature are not + negotiated, also Rx offload(s) are requested, this path will be selected. +#. Split virtqueue in-order mergeable path: If Rx mergeable and in-order feature are + both negotiated, this path will be selected. +#. Split virtqueue in-order non-mergeable path: If in-order feature is negotiated and + Rx mergeable is not negotiated, this path will be selected. +#. Split virtqueue vectorized Rx path: If Rx mergeable is disabled and no Rx offload + requested, this path will be selected. + +If packed virtqueue is negotiated, below packed virtqueue paths will be selected +according to below configuration: + +#. Packed virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature + is not negotiated, this path will be selected. +#. Packed virtqueue non-mergeable path: If Rx mergeable and in-order feature are not + negotiated, this path will be selected. +#. Packed virtqueue in-order mergeable path: If in-order and Rx mergeable feature are + both negotiated, this path will be selected. +#. Packed virtqueue in-order non-mergeable path: If in-order feature is negotiated and + Rx mergeable is not negotiated, this path will be selected. + +Rx/Tx callbacks of each Virtio path +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Refer to above description, virtio path and corresponding Rx/Tx callbacks will +be selected automatically. Rx callbacks and Tx callbacks for each virtio path +are shown in below table: + +.. table:: Virtio Paths and Callbacks + + ============================================ ================================= ======================== + Virtio paths Rx callbacks Tx callbacks + ============================================ ================================= ======================== + Split virtqueue mergeable path virtio_recv_mergeable_pkts virtio_xmit_pkts + Split virtqueue non-mergeable path virtio_recv_pkts virtio_xmit_pkts + Split virtqueue in-order mergeable path virtio_recv_pkts_inorder virtio_xmit_pkts_inorder + Split virtqueue in-order non-mergeable path virtio_recv_pkts_inorder virtio_xmit_pkts_inorder + Split virtqueue vectorized Rx path virtio_recv_pkts_vec virtio_xmit_pkts + Packed virtqueue mergeable path virtio_recv_mergeable_pkts_packed virtio_xmit_pkts_packed + Packed virtqueue non-meregable path virtio_recv_pkts_packed virtio_xmit_pkts_packed + Packed virtqueue in-order mergeable path virtio_recv_mergeable_pkts_packed virtio_xmit_pkts_packed + Packed virtqueue in-order non-mergeable path virtio_recv_pkts_packed virtio_xmit_pkts_packed + ============================================ ================================= ======================== + +Virtio paths Support Status from Release to Release +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Virtio feature implementation: + +* In-order feature is supported since DPDK 18.08 by adding new Rx/Tx callbacks + ``virtio_recv_pkts_inorder`` and ``virtio_xmit_pkts_inorder``. +* Packed virtqueue is supported since DPDK 19.02 by adding new Rx/Tx callbacks + ``virtio_recv_pkts_packed`` , ``virtio_recv_mergeable_pkts_packed`` and + ``virtio_xmit_pkts_packed``. + +All virtio paths support status are shown in below table: + +.. table:: Virtio Paths and Releases + + ============================================ ============= ============= ============= + Virtio paths 16.11 ~ 18.05 18.08 ~ 18.11 19.02 ~ 19.11 + ============================================ ============= ============= ============= + Split virtqueue mergeable path Y Y Y + Split virtqueue non-mergeable path Y Y Y + Split virtqueue vectorized Rx path Y Y Y + Split virtqueue simple Tx path Y N N + Split virtqueue in-order mergeable path Y Y + Split virtqueue in-order non-mergeable path Y Y + Packed virtqueue mergeable path Y + Packed virtqueue non-mergeable path Y + Packed virtqueue in-order mergeable path Y + Packed virtqueue in-order non-mergeable path Y + ============================================ ============= ============= ============= + +QEMU Support Status +~~~~~~~~~~~~~~~~~~~ + +* Qemu now supports three paths of split virtqueue: Split virtqueue mergeable path, + Split virtqueue non-mergeable path, Split virtqueue vectorized Rx path. +* Since qemu 4.2.0, Packed virtqueue mergeable path and Packed virtqueue non-mergeable + path can be supported. + +How to Debug +~~~~~~~~~~~~ + +If you meet performance drop or some other issues after upgrading the driver +or configuration, below steps can help you identify which path you selected and +root cause faster. + +#. Run vhost/virtio test case; +#. Run "perf top" and check virtio Rx/Tx callback names; +#. Identify which virtio path is selected refer to above table.
Signature · 1 line
2.21.0
Thomas Monjalon
Applied28/11/2019 15:30, Maxime Coquelin:
Quoted 8 lines
From: Yinan Wang <[email protected]>
Add virtio paths selection and usage introduction for better virtio usability.
Signed-off-by Yinan Wang <[email protected]>Acked-by Tiwei Bie <[email protected]>Reviewed-by Maxime Coquelin <[email protected]>
Applied, thanks