Skip to contentSkip to conversation

doc: clarify virtio PMD path selection

PATCH Superseded Yinan 3 participants 3 messages Nov 26, 2019

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Yinan

author
#0 lore raw

PATCH v3 1/1 doc: clarify virtio PMD path selection

From: Wang Yinan <[email protected]>

add virtio paths selection and usage introduction for better virtio usability.

---
 doc/guides/howto/index.rst                    |   1 +
 .../virtio_paths_selection_and_usage.rst      | 142 ++++++++++++++++++
 2 files changed, 143 insertions(+)
 create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rst
doc/guides/howto/index.rst +1 -0
index a4c131652..6edb8d5be 100644
@@ -16,6 +16,7 @@ HowTo Guides
     vfd
     virtio_user_for_container_networking
     virtio_user_as_exceptional_path
+    virtio_paths_selection_and_usage
     packet_capture_framework
     telemetry
     debug_troubleshoot
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +142 -0

142 lines not on this page · raw message

Signature · 1 line
2.17.1
#1 lore raw

Hi, yinan

Minor nit, please use '--in-reply-to' when using git-send-email, it helps tracking all history about this patch together.

Thanks,
Xiaolong

Quoted 175 lines · Yinan

From: Wang Yinan <[email protected]>

add virtio paths selection and usage introduction for better virtio usability.

---
doc/guides/howto/index.rst | 1 +
.../virtio_paths_selection_and_usage.rst | 142 ++++++++++++++++++

2 files changed, 143 insertions(+)

create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rst

doc/guides/howto/index.rst +1 -0
index a4c131652..6edb8d5be 100644
@@ -16,6 +16,7 @@ HowTo Guides
    vfd
    virtio_user_for_container_networking
    virtio_user_as_exceptional_path
+    virtio_paths_selection_and_usage
    packet_capture_framework
    telemetry
    debug_troubleshoot
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +142 -0
new file mode 100644
index 000000000..bfd933fac
@@ -0,0 +1,142 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Intel Corporation.
+
+Virtio paths Selection and Usage
+================================
+
+Logically virtio-PMD has 9 paths based on the combination of virtio features
+(Rx mergeable, In-order, Packed virtqueue), below is an introduction of virtio
+common features:
+
+*   `Rx mergeable <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/
+    virtio-v1.1-cs01.html#x1-700009>`_: With this feature negotiated, device
+    can receive larger packets by combining individual descriptors.
+*   `In-order <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/
+    virtio-v1.1-cs01.html#x1-690008>`_: Some devices always use descriptors
+    in the same order in which they have been made available, these
+    devices can offer the VIRTIO_F_IN_ORDER feature. If this feature negotiated,
+    driver will use descriptors in order. Meanwhile, this knowledge allows device
+    operate used ring in batches and driver operate available ring in batches and
+    such can decrease cache miss rate.
+*   `Packed virtqueue <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/
+    virtio-v1.1-cs01.html#x1-610007>`_: The structure of packed virtqueue is
+    different from split virtqueue, split virtqueue is composed of available ring,
+    used ring and descriptor table, while packed virtqueue is composed of descriptor
+    ring, driver event suppression and device event suppression. The idea behind
+    this is to improve performance by avoiding cache misses and and make it easier
+    for devices to implement.
+
+Virtio paths Selection
+----------------------
+
+If packed virtqueue is not negotiated, below split virtqueue paths can be selected
+according to below configuration:
+
+#. Split virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature is
+   not negotiated, this path will be selected.
+#. Split virtqueue non-mergeable path: If Rx mergeable and in-order feature are not
+   negotiated, also Rx offload(s) are requested, this path can be selected.
+#. Split virtqueue in-order mergeable path: If in-order feature and Rx mergeable are
+   both negotiated, this path can be selected.
+#. Split virtqueue in-order non-mergeable path: If in-order feature is negotiated and
+   Rx mergeable is not negotiated, this path can be selected.
+#. Split virtqueue vectorized RX path: If Rx mergeable is disabled and no Rx offload
+   requested, this path can be selected.
+
+If packed virtqueue is negotiated, below packed virtqueue paths can be selected
+according to below configuration:
+
+#. Packed virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature
+   is not negotiated, this path will be selected.
+#. Packed virtqueue non-mergeable path: If Rx mergeable and in-order feature are not
+   negotiated, also Rx offload(s) are requested, this path will be selected.
+#. Packed virtqueue in-order mergeable path: If in-order feature and Rx mergeable are
+   both negotiated, this path will be selected.
+#. Packed virtqueue in-order non-mergeable path: If in-order feature is negotiated and
+   Rx mergeable is not negotiated, this path will be selected.
+
+Rx/Tx callbacks of each Virtio path
+-----------------------------------
+
+Refer to above descriptions, virtio path and Rx/TX callbacks are auto selected by
+different parameters of vdev and workloads. Rx callbacks and Tx callbacks name for
+each Virtio Path are shown in following tables::
+
+   +----------------------------------------------------------------------------------------------------------+
+   |       Virtio path                          |   Rx callbacks                   |    TX callbacks          |
+   +----------------------------------------------------------------------------------------------------------+
+   |Split virtqueue mergeable path              |virtio_recv_mergeable_pkts        | virtio_xmit_pkts         |
+   +----------------------------------------------------------------------------------------------------------+
+   |Split virtqueue non-mergeable path          | virtio_recv_pkts                 |  virtio_xmit_pkts        |
+   +----------------------------------------------------------------------------------------------------------+
+   |Split virtqueue in-order mergeable path     | virtio_recv_pkts_inorder         |  virtio_xmit_pkts_inorder|
+   +----------------------------------------------------------------------------------------------------------+
+   |Split virtqueue in-order non-mergeable path | virtio_recv_pkts_inorder         |  virtio_xmit_pkts_inorder|
+   +----------------------------------------------------------------------------------------------------------+
+   |Split virtqueue vectorized RX path          | virtio_recv_pkts_vec             |  virtio_xmit_pkts        |
+   +----------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue mergeable path             | virtio_recv_mergeable_pkts_packed|  virtio_xmit_pkts_packed |
+   +----------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue normal path                | virtio_recv_pkts_packed          |  virtio_xmit_pkts_packed |
+   +----------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue in-order mergeable path    | virtio_recv_mergeable_pkts_packed|  virtio_xmit_pkts_packed |
+   +----------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue in-order normal path       | virtio_recv_pkts_packed          |  virtio_xmit_pkts_packed |
+   +----------------------------------------------------------------------------------------------------------+
+
+Virtio paths Support Status from Release to Release
+---------------------------------------------------
+
+Virtio feature implementation:
+
+*   In-order feature implemented in DPDK 18.08 by adding new Rx/TX callbacks
+    ``virtio_recv_pkts_inorder`` and ``virtio_xmit_pkts_inorder``.
+*   Packed virtqueue implemented in DPDK 19.02 by adding new Rx/TX callbacks
+    ``virtio_recv_pkts_packed`` , ``virtio_recv_mergeable_pkts_packed`` and
+    ``virtio_xmit_pkts_packed``.
+
+Virtio path number changes from release to release, all virtio paths support
+status are shown in below table::
+
+   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Virtio path\ DPDK version                   | v16.11 | v17.02 | v17.05 | v17.08 | v17.11 | v18.02 | v18.05 | v18.08 | v18.11 | v19.02 | v19.05 | v19.08 |
+   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue mergebale path              |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   Y    |   Y    |  Y     |   Y    |   Y    |
+   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue non-mergeable path          |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   Y    |   Y    |  Y     |   Y    |   Y    |
+   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue vectorized RX path          |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   Y    |   Y    |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue simple TX path              |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   N    |   N    |  N     |   N    |   N    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue in-order non-mergeable path |        |        |         |       |        |        |        |   Y    |   Y    |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue in-order mergeable path     |        |        |         |       |        |        |        |   Y    |   Y    |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue mergeable path             |        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue non-mergeable path         |        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue in-order mergeable path    |        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue in-order non-mergeable path|        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+
+QEMU Support Status
+-------------------
+
+*   Qemu now support three paths of split virtqueue: Split virtqueue mergebale path,
+    Split virtqueue no-mergeable path, Split virtqueue vectorized RX path.
+*   Since qemu 4.2.0, Packed virtqueue mergeable path and Packed virtqueue non-mergeable
+    path can be supported.
+
+How to Debug
+------------
+
+If you meet performance drop or some other issues after upgrading the driver
+or configuration, below steps can help you identify which path you selected and
+root cause faster.
+
+#. Run vhost/virtio test case;
+#. Run "perf top" and check virtio Rx/tx callback names;
+#. Identify which virtio path is selected refer to above table.
Signature · 1 line
2.17.1
#2 lore raw
In reply to Yinan

Hi Yinan,

Thanks for the work! Will take a closer look ASAP. Please see some quick comments inline.

Quoted 7 lines · Yinan

From: Wang Yinan <[email protected]>

add virtio paths selection and usage introduction for better virtio usability.

---

When sending new versions, please use --in-reply-to [1]. And it would be better if you can add a brief changelog.

[1] https://core.dpdk.org/contribute/#send

Quoted 25 lines
 doc/guides/howto/index.rst                    |   1 +
 .../virtio_paths_selection_and_usage.rst      | 142 ++++++++++++++++++
 2 files changed, 143 insertions(+)
 create mode 100644 doc/guides/howto/virtio_paths_selection_and_usage.rst
doc/guides/howto/index.rst +1 -0
index a4c131652..6edb8d5be 100644
@@ -16,6 +16,7 @@ HowTo Guides
     vfd
     virtio_user_for_container_networking
     virtio_user_as_exceptional_path
+    virtio_paths_selection_and_usage
     packet_capture_framework
     telemetry
     debug_troubleshoot
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +2 -0
new file mode 100644
index 000000000..bfd933fac
@@ -0,0 +1,142 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Intel Corporation.

doc/guides/howto/virtio_paths_selection_and_usage.rst line 12

Quoted 10 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +10 -0
+
+Virtio paths Selection and Usage
+================================
+
+Logically virtio-PMD has 9 paths based on the combination of virtio features
+(Rx mergeable, In-order, Packed virtqueue), below is an introduction of virtio
+common features:
+
+*   `Rx mergeable <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/
+    virtio-v1.1-cs01.html#x1-700009>`_: With this feature negotiated, device

Instead of referring to "Multi-buffer requests", below section is more related:

https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/virtio-v1.1-cs01.html#x1-2140004

Quoted 16 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +16 -0
+    can receive larger packets by combining individual descriptors.
+*   `In-order <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/
+    virtio-v1.1-cs01.html#x1-690008>`_: Some devices always use descriptors
+    in the same order in which they have been made available, these
+    devices can offer the VIRTIO_F_IN_ORDER feature. If this feature negotiated,
+    driver will use descriptors in order. Meanwhile, this knowledge allows device
+    operate used ring in batches and driver operate available ring in batches and
+    such can decrease cache miss rate.
+*   `Packed virtqueue <https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/
+    virtio-v1.1-cs01.html#x1-610007>`_: The structure of packed virtqueue is
+    different from split virtqueue, split virtqueue is composed of available ring,
+    used ring and descriptor table, while packed virtqueue is composed of descriptor
+    ring, driver event suppression and device event suppression. The idea behind
+    this is to improve performance by avoiding cache misses and and make it easier
+    for devices to implement.
+

doc/guides/howto/virtio_paths_selection_and_usage.rst line 38

Quoted 10 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +10 -0
+Virtio paths Selection
+----------------------
+
+If packed virtqueue is not negotiated, below split virtqueue paths can be selected
+according to below configuration:
+
+#. Split virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature is
+   not negotiated, this path will be selected.
+#. Split virtqueue non-mergeable path: If Rx mergeable and in-order feature are not
+   negotiated, also Rx offload(s) are requested, this path can be selected.

s/can be/will be/
This applies to other similar cases as well.

Quoted 4 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +4 -0
+#. Split virtqueue in-order mergeable path: If in-order feature and Rx mergeable are
+   both negotiated, this path can be selected.
+#. Split virtqueue in-order non-mergeable path: If in-order feature is negotiated and
+   Rx mergeable is not negotiated, this path can be selected.

doc/guides/howto/virtio_paths_selection_and_usage.rst line 52

Quoted 10 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +10 -0
+#. Split virtqueue vectorized RX path: If Rx mergeable is disabled and no Rx offload
+   requested, this path can be selected.
+
+If packed virtqueue is negotiated, below packed virtqueue paths can be selected
+according to below configuration:
+
+#. Packed virtqueue mergeable path: If Rx mergeable is negotiated, in-order feature
+   is not negotiated, this path will be selected.
+#. Packed virtqueue non-mergeable path: If Rx mergeable and in-order feature are not
+   negotiated, also Rx offload(s) are requested, this path will be selected.

Currently, requesting Rx offload or not doesn't matter in this case.

Quoted 42 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +42 -0
+#. Packed virtqueue in-order mergeable path: If in-order feature and Rx mergeable are
+   both negotiated, this path will be selected.
+#. Packed virtqueue in-order non-mergeable path: If in-order feature is negotiated and
+   Rx mergeable is not negotiated, this path will be selected.
+
+Rx/Tx callbacks of each Virtio path
+-----------------------------------
+
+Refer to above descriptions, virtio path and Rx/TX callbacks are auto selected by
+different parameters of vdev and workloads. Rx callbacks and Tx callbacks name for
+each Virtio Path are shown in following tables::
+
+   +----------------------------------------------------------------------------------------------------------+
+   |       Virtio path                          |   Rx callbacks                   |    TX callbacks          |
+   +----------------------------------------------------------------------------------------------------------+
+   |Split virtqueue mergeable path              |virtio_recv_mergeable_pkts        | virtio_xmit_pkts         |
+   +----------------------------------------------------------------------------------------------------------+
+   |Split virtqueue non-mergeable path          | virtio_recv_pkts                 |  virtio_xmit_pkts        |
+   +----------------------------------------------------------------------------------------------------------+
+   |Split virtqueue in-order mergeable path     | virtio_recv_pkts_inorder         |  virtio_xmit_pkts_inorder|
+   +----------------------------------------------------------------------------------------------------------+
+   |Split virtqueue in-order non-mergeable path | virtio_recv_pkts_inorder         |  virtio_xmit_pkts_inorder|
+   +----------------------------------------------------------------------------------------------------------+
+   |Split virtqueue vectorized RX path          | virtio_recv_pkts_vec             |  virtio_xmit_pkts        |
+   +----------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue mergeable path             | virtio_recv_mergeable_pkts_packed|  virtio_xmit_pkts_packed |
+   +----------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue normal path                | virtio_recv_pkts_packed          |  virtio_xmit_pkts_packed |
+   +----------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue in-order mergeable path    | virtio_recv_mergeable_pkts_packed|  virtio_xmit_pkts_packed |
+   +----------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue in-order normal path       | virtio_recv_pkts_packed          |  virtio_xmit_pkts_packed |
+   +----------------------------------------------------------------------------------------------------------+
+
+Virtio paths Support Status from Release to Release
+---------------------------------------------------
+
+Virtio feature implementation:
+
+*   In-order feature implemented in DPDK 18.08 by adding new Rx/TX callbacks
+    ``virtio_recv_pkts_inorder`` and ``virtio_xmit_pkts_inorder``.
+*   Packed virtqueue implemented in DPDK 19.02 by adding new Rx/TX callbacks

doc/guides/howto/virtio_paths_selection_and_usage.rst line 104

Quoted 10 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +10 -0
+    ``virtio_recv_pkts_packed`` , ``virtio_recv_mergeable_pkts_packed`` and
+    ``virtio_xmit_pkts_packed``.
+
+Virtio path number changes from release to release, all virtio paths support
+status are shown in below table::
+
+   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Virtio path\ DPDK version                   | v16.11 | v17.02 | v17.05 | v17.08 | v17.11 | v18.02 | v18.05 | v18.08 | v18.11 | v19.02 | v19.05 | v19.08 |
+   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue mergebale path              |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   Y    |   Y    |  Y     |   Y    |   Y    |

s/mergebale/mergeable/

Quoted 14 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +14 -0
+   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue non-mergeable path          |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   Y    |   Y    |  Y     |   Y    |   Y    |
+   +--------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue vectorized RX path          |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   Y    |   Y    |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue simple TX path              |   Y    |   Y    |    Y    |   Y   |   Y    |   Y    |   Y    |   N    |   N    |  N     |   N    |   N    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue in-order non-mergeable path |        |        |         |       |        |        |        |   Y    |   Y    |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Split virtqueue in-order mergeable path     |        |        |         |       |        |        |        |   Y    |   Y    |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue mergeable path             |        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue non-mergeable path         |        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |

doc/guides/howto/virtio_paths_selection_and_usage.rst line 128

Quoted 10 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +10 -0
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue in-order mergeable path    |        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+   |Packed virtqueue in-order non-mergeable path|        |        |         |       |        |        |        |        |        |  Y     |   Y    |   Y    |
+   ---------------------------------------------------------------------------------------------------------------------------------------------------------+
+
+QEMU Support Status
+-------------------
+
+*   Qemu now support three paths of split virtqueue: Split virtqueue mergebale path,

s/support/supports/
s/mergebale/mergeable/

doc/guides/howto/virtio_paths_selection_and_usage.rst line 129

Quoted 1 line
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +1 -0
+    Split virtqueue no-mergeable path, Split virtqueue vectorized RX path.

s/no-mergeable/non-mergeable/

Quoted 16 lines
doc/guides/howto/virtio_paths_selection_and_usage.rst new file +13 -0
+*   Since qemu 4.2.0, Packed virtqueue mergeable path and Packed virtqueue non-mergeable
+    path can be supported.
+
+How to Debug
+------------
+
+If you meet performance drop or some other issues after upgrading the driver
+or configuration, below steps can help you identify which path you selected and
+root cause faster.
+
+#. Run vhost/virtio test case;
+#. Run "perf top" and check virtio Rx/tx callback names;
+#. Identify which virtio path is selected refer to above table.
Signature · 1 line
2.17.1