update docs for vhost async API
cover update docs for vhost async API
From: Patrick Fu <[email protected]>
update documentations for vhost async API
Patrick Fu (2):
doc: update guides for vhost async APIs
doc: update release notes for vhost async APIs
doc/guides/prog_guide/vhost_lib.rst | 86 +++++++++++++++++++++++--- doc/guides/rel_notes/release_20_08.rst | 5 ++ 2 files changed, 82 insertions(+), 9 deletions(-)
Signature · 1 line
2.18.4
PATCH 1/2 doc: update guides for vhost async APIs
From: Patrick Fu <[email protected]>
Update vhost guides to document vhost async APIs
--- doc/guides/prog_guide/vhost_lib.rst | 86 ++++++++++++++++++++++++++--- 1 file changed, 77 insertions(+), 9 deletions(-)
doc/guides/prog_guide/vhost_lib.rst +77 -9
@@ -147,6 +147,21 @@ The following is an overview of some key Vhost API functions: It is disabled by default. + - ``RTE_VHOST_USER_ASYNC_COPY`` + + Asynchronous data path will be enabled when this flag is set. Async data + path allows applications to register async copy devices (typically + hardware DMA channels) to the vhost queues. Vhost leverages the copy + device registered to offload CPU from memory copy operations. A set of + async data path APIs are defined for DPDK applications to make use of + the async capability. Only packets enqueued/dequeued by async APIs are + processed through the async data path. + + Currently this feature is only implemented on split ring enqueue data + path. + + It is disabled by default. + * ``rte_vhost_driver_set_features(path, features)`` This function sets the feature bits the vhost-user driver supports. The
@@ -235,6 +250,59 @@ The following is an overview of some key Vhost API functions: Enable or disable zero copy feature of the vhost crypto backend. +* ``rte_vhost_async_channel_register(vid, queue_id, features, ops)`` + + Register a vhost queue with async copy device channel. + Following device ``features`` must be specified together with the + registration: + + * ``async_inorder`` + + Async copy device can guarantee the ordering of copy completion + sequence. Copies are completed in the same order with that at + the submission time. + + Currently, only ``async_inorder`` capable device is supported by vhost. + + * ``async_threshold`` + + The copy length (in bytes) below which CPU copy will be used even if + applications call async vhost APIs to enqueue/dequeue data. + + Typical value is 512~1024 depending on the async device capability. + + Applications must provide following ``ops`` callbacks for vhost lib to + work with the async copye devices: + + * ``transfer_data(vid, queue_id, descs, opaque_data, count)`` + + vhost invokes this function to submit copy data to the async devices. + For non-async_inorder capable devices, ``opaque_data`` could be used + for identifying the completed packets. + + * ``check_completed_copies(vid, queue_id, opaque_data, max_packets)`` + + vhost invokes this function to get the copy data completed by async + devices. + +* ``rte_vhost_async_channel_unregister(vid, queue_id)`` + + Unregister the async copy device from a vhost queue. + +* ``rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count)`` + + Submit an enqueue request to transmit ``count`` packets from host to guest + by async data path. Enqueue is not guaranteed to finish upon the return of + this API call. + + Applications must not free the packets submitted for enqueue until the + packets are completed. + +* ``rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count)`` + + Poll enqueue completion status from async data path. Completed packets + are returned to applications through ``pkts``. + Vhost-user Implementations --------------------------
@@ -294,16 +362,16 @@ Guest memory requirement * Memory pre-allocation - For non-zerocopy, guest memory pre-allocation is not a must. This can help - save of memory. If users really want the guest memory to be pre-allocated - (e.g., for performance reason), we can add option ``-mem-prealloc`` when - starting QEMU. Or, we can lock all memory at vhost side which will force - memory to be allocated when mmap at vhost side; option --mlockall in - ovs-dpdk is an example in hand. + For non-zerocopy non-async data path, guest memory pre-allocation is not a + must. This can help save of memory. If users really want the guest memory + to be pre-allocated (e.g., for performance reason), we can add option + ``-mem-prealloc`` when starting QEMU. Or, we can lock all memory at vhost + side which will force memory to be allocated when mmap at vhost side; + option --mlockall in ovs-dpdk is an example in hand. - For zerocopy, we force the VM memory to be pre-allocated at vhost lib when - mapping the guest memory; and also we need to lock the memory to prevent - pages being swapped out to disk. + For async data path or zerocopy, we force the VM memory to be pre-allocated + at vhost lib when mapping the guest memory; and also we need to lock the + memory to prevent pages being swapped out to disk. * Memory sharing
Signature · 1 line
2.18.4
Re: [dpdk-dev] [PATCH v1 1/2] doc: update guides for vhost async APIs
Hi Patrick,
Quoted 27 lines
-----Original Message-----
From: Fu, Patrick <[email protected]>
Sent: Wednesday, July 22, 2020 6:58 PM
To: [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Fu, Patrick <[email protected]>
Subject: [PATCH v1 1/2] doc: update guides for vhost async APIsFrom: Patrick Fu <[email protected]>
Update vhost guides to document vhost async APIs
Signed-off-by Patrick Fu <[email protected]>--- doc/guides/prog_guide/vhost_lib.rst | 86 ++++++++++++++++++++++++++--- 1 file changed, 77 insertions(+), 9 deletions(-)doc/guides/prog_guide/vhost_lib.rst +0
-0@@ -147,6 +147,21 @@ The following is an overview of some key Vhost APIfunctions:
It is disabled by default.
doc/guides/prog_guide/vhost_lib.rst The following is an overview of some key Vhost API functions: · line 155
Quoted 6 lines
doc/guides/prog_guide/vhost_lib.rst +6
-0+ - ``RTE_VHOST_USER_ASYNC_COPY`` + + Asynchronous data path will be enabled when this flag is set. Async data + path allows applications to register async copy devices (typically + hardware DMA channels) to the vhost queues. Vhost leverages the copy + device registered to offload CPU from memory copy operations. A set of
You mean 'free' CPU from memory copy?
Quoted 30 lines
doc/guides/prog_guide/vhost_lib.rst +9
-0+ async data path APIs are defined for DPDK applications to make use of + the async capability. Only packets enqueued/dequeued by async APIs are + processed through the async data path. + + Currently this feature is only implemented on split ring enqueue data + path. + + It is disabled by default. + * ``rte_vhost_driver_set_features(path, features)`` This function sets the feature bits the vhost-user driver supports. The @@ -235,6 +250,59 @@ The following is an overview of some key Vhost API functions:
Enable or disable zero copy feature of the vhost crypto backend.
doc/guides/prog_guide/vhost_lib.rst +13
-0+* ``rte_vhost_async_channel_register(vid, queue_id, features, ops)`` + + Register a vhost queue with async copy device channel. + Following device ``features`` must be specified together with the + registration: + + * ``async_inorder`` + + Async copy device can guarantee the ordering of copy completion + sequence. Copies are completed in the same order with that at + the submission time. + + Currently, only ``async_inorder`` capable device is supported by vhost.
doc/guides/prog_guide/vhost_lib.rst The following is an overview of some key Vhost API functions:
Quoted 10 lines
doc/guides/prog_guide/vhost_lib.rst +10
-0+ + * ``async_threshold`` + + The copy length (in bytes) below which CPU copy will be used even if + applications call async vhost APIs to enqueue/dequeue data. + + Typical value is 512~1024 depending on the async device capability. + + Applications must provide following ``ops`` callbacks for vhost lib + to work with the async copye devices:
s/copye/copy
Quoted 5 lines
doc/guides/prog_guide/vhost_lib.rst +5
-0+ + * ``transfer_data(vid, queue_id, descs, opaque_data, count)`` + + vhost invokes this function to submit copy data to the async devices. + For non-async_inorder capable devices, ``opaque_data`` could be used
doc/guides/prog_guide/vhost_lib.rst The following is an overview of some key Vhost API functions: · line 290
Quoted 10 lines
doc/guides/prog_guide/vhost_lib.rst +10
-0+ for identifying the completed packets. + + * ``check_completed_copies(vid, queue_id, opaque_data, max_packets)`` + + vhost invokes this function to get the copy data completed by async + devices. + +* ``rte_vhost_async_channel_unregister(vid, queue_id)`` + + Unregister the async copy device from a vhost queue.
'Copy device channel' may be more accurate?
Quoted 29 lines
doc/guides/prog_guide/vhost_lib.rst +16
-6+ +* ``rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count)`` + + Submit an enqueue request to transmit ``count`` packets from host to + guest by async data path. Enqueue is not guaranteed to finish upon + the return of this API call. + + Applications must not free the packets submitted for enqueue until + the packets are completed. + +* ``rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count)`` + + Poll enqueue completion status from async data path. Completed + packets are returned to applications through ``pkts``. + Vhost-user Implementations --------------------------@@ -294,16 +362,16 @@ Guest memory requirement * Memory pre-allocation - For non-zerocopy, guest memory pre-allocation is not a must. This can help - save of memory. If users really want the guest memory to be pre-allocated - (e.g., for performance reason), we can add option ``-mem-prealloc`` when - starting QEMU. Or, we can lock all memory at vhost side which will force - memory to be allocated when mmap at vhost side; option --mlockall in - ovs-dpdk is an example in hand. + For non-zerocopy non-async data path, guest memory pre-allocation is
doc/guides/prog_guide/vhost_lib.rst Guest memory requirement · line 372
Quoted 10 lines
doc/guides/prog_guide/vhost_lib.rst +6
-3+ not a must. This can help save of memory. If users really want the + guest memory to be pre-allocated (e.g., for performance reason), we + can add option ``-mem-prealloc`` when starting QEMU. Or, we can lock + all memory at vhost side which will force memory to be allocated when + mmap at vhost side; option --mlockall in ovs-dpdk is an example in hand. - For zerocopy, we force the VM memory to be pre-allocated at vhost lib when - mapping the guest memory; and also we need to lock the memory to prevent - pages being swapped out to disk. + For async data path or zerocopy, we force the VM memory to be
'For async or zerocopy data path' may be better?
Thanks!
Chenbo
Quoted 7 lines
doc/guides/prog_guide/vhost_lib.rst +2
-1+ pre-allocated at vhost lib when mapping the guest memory; and also we + need to lock the memory to prevent pages being swapped out to disk. * Memory sharing --2.18.4
Re: [dpdk-dev] [PATCH v1 1/2] doc: update guides for vhost async APIs
Thanks for comments. v2 patch sent with all the changes suggested.
Thanks,
Patrick
Quoted 173 lines
-----Original Message-----
From: Xia, Chenbo <[email protected]>
Sent: Wednesday, July 22, 2020 7:21 PM
To: Fu, Patrick <[email protected]>; [email protected]; [email protected]
Subject: RE: [PATCH v1 1/2] doc: update guides for vhost async APIsHi Patrick,
Quoted 29 lines
-----Original Message-----
From: Fu, Patrick <[email protected]>
Sent: Wednesday, July 22, 2020 6:58 PM
To: [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Fu, Patrick <[email protected]>
Subject: [PATCH v1 1/2] doc: update guides for vhost async APIsFrom: Patrick Fu <[email protected]>
Update vhost guides to document vhost async APIs
Signed-off-by Patrick Fu <[email protected]>--- doc/guides/prog_guide/vhost_lib.rst | 86 ++++++++++++++++++++++++++--- 1 file changed, 77 insertions(+), 9 deletions(-)doc/guides/prog_guide/vhost_lib.rst +0
-0@@ -147,6 +147,21 @@ The following is an overview of some key VhostAPI
functions:It is disabled by default.
doc/guides/prog_guide/vhost_lib.rst The following is an overview of some key Vhost API functions:
Quoted 3 lines
doc/guides/prog_guide/vhost_lib.rst +3
-0+ - ``RTE_VHOST_USER_ASYNC_COPY`` + + Asynchronous data path will be enabled when this flag is set. Asyncdata
doc/guides/prog_guide/vhost_lib.rst The following is an overview of some key Vhost API functions:
Quoted 4 lines
doc/guides/prog_guide/vhost_lib.rst +4
-0+ path allows applications to register async copy devices (typically + hardware DMA channels) to the vhost queues. Vhost leverages the copy + device registered to offload CPU from memory copy operations. A + set ofYou mean 'free' CPU from memory copy?
doc/guides/prog_guide/vhost_lib.rst
Quoted 2 lines
doc/guides/prog_guide/vhost_lib.rst +2
-0+ async data path APIs are defined for DPDK applications to make use of + the async capability. Only packets enqueued/dequeued by async APIsare
Quoted 29 lines
doc/guides/prog_guide/vhost_lib.rst +7
-0+ processed through the async data path. + + Currently this feature is only implemented on split ring enqueue data + path. + + It is disabled by default. + * ``rte_vhost_driver_set_features(path, features)`` This function sets the feature bits the vhost-user driver supports.The @@ -
235,6 +250,59 @@ The following is an overview of some key Vhost API functions:Enable or disable zero copy feature of the vhost crypto backend.
doc/guides/prog_guide/vhost_lib.rst +13
-0+* ``rte_vhost_async_channel_register(vid, queue_id, features, ops)`` + + Register a vhost queue with async copy device channel. + Following device ``features`` must be specified together with the + registration: + + * ``async_inorder`` + + Async copy device can guarantee the ordering of copy completion + sequence. Copies are completed in the same order with that at + the submission time. + + Currently, only ``async_inorder`` capable device is supported by vhost.doc/guides/prog_guide/vhost_lib.rst The following is an overview of some key Vhost API functions:
Quoted 10 lines
doc/guides/prog_guide/vhost_lib.rst +10
-0+ + * ``async_threshold`` + + The copy length (in bytes) below which CPU copy will be used even if + applications call async vhost APIs to enqueue/dequeue data. + + Typical value is 512~1024 depending on the async device capability. + + Applications must provide following ``ops`` callbacks for vhost lib + to work with the async copye devices:s/copye/copy
Quoted 6 lines
doc/guides/prog_guide/vhost_lib.rst +6
-0+ + * ``transfer_data(vid, queue_id, descs, opaque_data, count)`` + + vhost invokes this function to submit copy data to the async devices. + For non-async_inorder capable devices, ``opaque_data`` could be used + for identifying the completed packets.doc/guides/prog_guide/vhost_lib.rst The following is an overview of some key Vhost API functions: · line 290
Quoted 10 lines
doc/guides/prog_guide/vhost_lib.rst +10
-0+ + * ``check_completed_copies(vid, queue_id, opaque_data, + max_packets)`` + + vhost invokes this function to get the copy data completed by async + devices. + +* ``rte_vhost_async_channel_unregister(vid, queue_id)`` + + Unregister the async copy device from a vhost queue.'Copy device channel' may be more accurate?
Quoted 32 lines
doc/guides/prog_guide/vhost_lib.rst +15
-1+ +* ``rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count)`` + + Submit an enqueue request to transmit ``count`` packets from host + to guest by async data path. Enqueue is not guaranteed to finish + upon the return of this API call. + + Applications must not free the packets submitted for enqueue until + the packets are completed. + +* ``rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count)`` + + Poll enqueue completion status from async data path. Completed + packets are returned to applications through ``pkts``. + Vhost-user Implementations --------------------------@@ -294,16 +362,16 @@ Guest memory requirement * Memory pre-allocation - For non-zerocopy, guest memory pre-allocation is not a must. Thiscan help
- save of memory. If users really want the guest memory to be pre-allocated
- (e.g., for performance reason), we can add option ``-mem-prealloc`` when
- starting QEMU. Or, we can lock all memory at vhost side which will force
- memory to be allocated when mmap at vhost side; option --mlockall indoc/guides/prog_guide/vhost_lib.rst
Quoted 7 lines
doc/guides/prog_guide/vhost_lib.rst +6
-1- ovs-dpdk is an example in hand. + For non-zerocopy non-async data path, guest memory pre-allocation + is not a must. This can help save of memory. If users really want + the guest memory to be pre-allocated (e.g., for performance + reason), we can add option ``-mem-prealloc`` when starting QEMU. + Or, we can lock all memory at vhost side which will force memory to + be allocated when mmap at vhost side; option --mlockall in ovs-dpdk isan example in hand.
doc/guides/prog_guide/vhost_lib.rst
Quoted 7 lines
- For zerocopy, we force the VM memory to be pre-allocated at vhost lib when
- mapping the guest memory; and also we need to lock the memory to preventdoc/guides/prog_guide/vhost_lib.rst +1
-1- pages being swapped out to disk. + For async data path or zerocopy, we force the VM memory to be'For async or zerocopy data path' may be better?
Thanks!
ChenboQuoted 7 lines
+ pre-allocated at vhost lib when mapping the guest memory; and also
+ we need to lock the memory to prevent pages being swapped out to disk.* Memory sharing
--
2.18.4
PATCH 2/2 doc: update release notes for vhost async APIs
In reply to [email protected]From: Patrick Fu <[email protected]>
Update 20.08 release notes to include vhost async APIs
--- doc/guides/rel_notes/release_20_08.rst | 5 +++++ 1 file changed, 5 insertions(+)
doc/guides/rel_notes/release_20_08.rst +5 -0
@@ -225,6 +225,11 @@ New Features See the :doc:`../sample_app_ug/l2_forward_real_virtual` for more details of this parameter usage. +* **Added vhost async data path APIs.** + + 4 new APIs has been added to enable vhost async data path. With the new + data path, applications can register vhost with copy devices, such as DMA + engines, to offload memory copy operations. Removed Items -------------
Signature · 1 line
2.18.4
Re: [dpdk-dev] [PATCH v1 2/2] doc: update release notes for vhost async APIs
Hi Patrick,
Quoted 23 lines
-----Original Message-----
From: Fu, Patrick <[email protected]>
Sent: Wednesday, July 22, 2020 6:58 PM
To: [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Fu, Patrick <[email protected]>
Subject: [PATCH v1 2/2] doc: update release notes for vhost async APIsFrom: Patrick Fu <[email protected]>
Update 20.08 release notes to include vhost async APIs
Signed-off-by Patrick Fu <[email protected]>--- doc/guides/rel_notes/release_20_08.rst | 5 +++++ 1 file changed, 5 insertions(+)doc/guides/rel_notes/release_20_08.rst +0
-0
doc/guides/rel_notes/release_20_08.rst New Features · line 230
Quoted 6 lines
doc/guides/rel_notes/release_20_08.rst +3
-0@@ -225,6 +225,11 @@ New Features See the :doc:`../sample_app_ug/l2_forward_real_virtual` for more details of this parameter usage. +* **Added vhost async data path APIs.** + + 4 new APIs has been added to enable vhost async data path. With the
s/has/have
Should we mention you only added enqueue support?
Thanks,
Chenbo
Quoted 7 lines
doc/guides/rel_notes/release_20_08.rst +2
-1+ new data path, applications can register vhost with copy devices, + such as DMA engines, to offload memory copy operations. Removed Items ------------- --2.18.4
Re: [dpdk-dev] [PATCH v1 2/2] doc: update release notes for vhost async APIs
Hi,
Quoted 35 lines
-----Original Message-----
From: Xia, Chenbo <[email protected]>
Sent: Wednesday, July 22, 2020 7:25 PM
To: Fu, Patrick <[email protected]>; [email protected]; [email protected]
Subject: RE: [PATCH v1 2/2] doc: update release notes for vhost async APIsHi Patrick,
Quoted 26 lines
-----Original Message-----
From: Fu, Patrick <[email protected]>
Sent: Wednesday, July 22, 2020 6:58 PM
To: [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Fu, Patrick <[email protected]>
Subject: [PATCH v1 2/2] doc: update release notes for vhost async APIsFrom: Patrick Fu <[email protected]>
Update 20.08 release notes to include vhost async APIs
Signed-off-by Patrick Fu <[email protected]>--- doc/guides/rel_notes/release_20_08.rst | 5 +++++ 1 file changed, 5 insertions(+)doc/guides/rel_notes/release_20_08.rst +0
-0@@ -225,6 +225,11 @@ New Features See the :doc:`../sample_app_ug/l2_forward_real_virtual` for more details of this parameter usage.
Quoted 7 lines
doc/guides/rel_notes/release_20_08.rst New Features
Quoted 3 lines
doc/guides/rel_notes/release_20_08.rst +3
-0+* **Added vhost async data path APIs.** + + 4 new APIs has been added to enable vhost async data path. With thes/has/have
Should we mention you only added enqueue support?
I included more details in v2 patch for the release notes.
Thanks,
Patrick
v2 cover update docs for vhost async APIs
In reply to [email protected]From: Patrick Fu <[email protected]>
update docs for vhost async APIs
v2:
- fix typos in vhost_lib.rst
- minor rewording in vhost_lib.rst
- add more detailed API notes in release notes
- rebase the patch to 20.08 rc2
Patrick Fu (2):
doc: update guides for vhost async APIs
doc: update release notes for vhost async APIs
doc/guides/prog_guide/vhost_lib.rst | 86 +++++++++++++++++++++++--- doc/guides/rel_notes/release_20_08.rst | 6 ++ 2 files changed, 83 insertions(+), 9 deletions(-)
Signature · 1 line
2.18.4
PATCH v2 1/2 doc: update guides for vhost async APIs
From: Patrick Fu <[email protected]>
Update vhost guides to document vhost async APIs
--- doc/guides/prog_guide/vhost_lib.rst | 86 ++++++++++++++++++++++++++--- 1 file changed, 77 insertions(+), 9 deletions(-)
doc/guides/prog_guide/vhost_lib.rst +77 -9
@@ -147,6 +147,21 @@ The following is an overview of some key Vhost API functions: It is disabled by default. + - ``RTE_VHOST_USER_ASYNC_COPY`` + + Asynchronous data path will be enabled when this flag is set. Async data + path allows applications to register async copy devices (typically + hardware DMA channels) to the vhost queues. Vhost leverages the copy + device registered to free CPU from memory copy operations. A set of + async data path APIs are defined for DPDK applications to make use of + the async capability. Only packets enqueued/dequeued by async APIs are + processed through the async data path. + + Currently this feature is only implemented on split ring enqueue data + path. + + It is disabled by default. + * ``rte_vhost_driver_set_features(path, features)`` This function sets the feature bits the vhost-user driver supports. The
@@ -235,6 +250,59 @@ The following is an overview of some key Vhost API functions: Enable or disable zero copy feature of the vhost crypto backend. +* ``rte_vhost_async_channel_register(vid, queue_id, features, ops)`` + + Register a vhost queue with async copy device channel. + Following device ``features`` must be specified together with the + registration: + + * ``async_inorder`` + + Async copy device can guarantee the ordering of copy completion + sequence. Copies are completed in the same order with that at + the submission time. + + Currently, only ``async_inorder`` capable device is supported by vhost. + + * ``async_threshold`` + + The copy length (in bytes) below which CPU copy will be used even if + applications call async vhost APIs to enqueue/dequeue data. + + Typical value is 512~1024 depending on the async device capability. + + Applications must provide following ``ops`` callbacks for vhost lib to + work with the async copy devices: + + * ``transfer_data(vid, queue_id, descs, opaque_data, count)`` + + vhost invokes this function to submit copy data to the async devices. + For non-async_inorder capable devices, ``opaque_data`` could be used + for identifying the completed packets. + + * ``check_completed_copies(vid, queue_id, opaque_data, max_packets)`` + + vhost invokes this function to get the copy data completed by async + devices. + +* ``rte_vhost_async_channel_unregister(vid, queue_id)`` + + Unregister the async copy device channel from a vhost queue. + +* ``rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count)`` + + Submit an enqueue request to transmit ``count`` packets from host to guest + by async data path. Enqueue is not guaranteed to finish upon the return of + this API call. + + Applications must not free the packets submitted for enqueue until the + packets are completed. + +* ``rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count)`` + + Poll enqueue completion status from async data path. Completed packets + are returned to applications through ``pkts``. + Vhost-user Implementations --------------------------
@@ -294,16 +362,16 @@ Guest memory requirement * Memory pre-allocation - For non-zerocopy, guest memory pre-allocation is not a must. This can help - save of memory. If users really want the guest memory to be pre-allocated - (e.g., for performance reason), we can add option ``-mem-prealloc`` when - starting QEMU. Or, we can lock all memory at vhost side which will force - memory to be allocated when mmap at vhost side; option --mlockall in - ovs-dpdk is an example in hand. + For non-zerocopy non-async data path, guest memory pre-allocation is not a + must. This can help save of memory. If users really want the guest memory + to be pre-allocated (e.g., for performance reason), we can add option + ``-mem-prealloc`` when starting QEMU. Or, we can lock all memory at vhost + side which will force memory to be allocated when mmap at vhost side; + option --mlockall in ovs-dpdk is an example in hand. - For zerocopy, we force the VM memory to be pre-allocated at vhost lib when - mapping the guest memory; and also we need to lock the memory to prevent - pages being swapped out to disk. + For async and zerocopy data path, we force the VM memory to be + pre-allocated at vhost lib when mapping the guest memory; and also we need + to lock the memory to prevent pages being swapped out to disk. * Memory sharing
Signature · 1 line
2.18.4
Re: [dpdk-dev] [PATCH v2 1/2] doc: update guides for vhost async APIs
Quoted 134 lines
-----Original Message-----
From: Fu, Patrick <[email protected]>
Sent: Wednesday, July 22, 2020 11:02 PM
To: [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Fu, Patrick <[email protected]>
Subject: [PATCH v2 1/2] doc: update guides for vhost async APIsFrom: Patrick Fu <[email protected]>
Update vhost guides to document vhost async APIs
Signed-off-by Patrick Fu <[email protected]>--- doc/guides/prog_guide/vhost_lib.rst | 86 ++++++++++++++++++++++++++--- 1 file changed, 77 insertions(+), 9 deletions(-)doc/guides/prog_guide/vhost_lib.rst +0
-0@@ -147,6 +147,21 @@ The following is an overview of some key Vhost APIfunctions:
It is disabled by default.
doc/guides/prog_guide/vhost_lib.rst +15
-0+ - ``RTE_VHOST_USER_ASYNC_COPY`` + + Asynchronous data path will be enabled when this flag is set. Async data + path allows applications to register async copy devices (typically + hardware DMA channels) to the vhost queues. Vhost leverages the copy + device registered to free CPU from memory copy operations. A set of + async data path APIs are defined for DPDK applications to make use of + the async capability. Only packets enqueued/dequeued by async APIs are + processed through the async data path. + + Currently this feature is only implemented on split ring enqueue data + path. + + It is disabled by default. + * ``rte_vhost_driver_set_features(path, features)`` This function sets the feature bits the vhost-user driver supports. The @@ -235,6 +250,59 @@ The following is an overview of some key Vhost API functions:
Enable or disable zero copy feature of the vhost crypto backend.
doc/guides/prog_guide/vhost_lib.rst +62
-10+* ``rte_vhost_async_channel_register(vid, queue_id, features, ops)`` + + Register a vhost queue with async copy device channel. + Following device ``features`` must be specified together with the + registration: + + * ``async_inorder`` + + Async copy device can guarantee the ordering of copy completion + sequence. Copies are completed in the same order with that at + the submission time. + + Currently, only ``async_inorder`` capable device is supported by vhost. + + * ``async_threshold`` + + The copy length (in bytes) below which CPU copy will be used even if + applications call async vhost APIs to enqueue/dequeue data. + + Typical value is 512~1024 depending on the async device capability. + + Applications must provide following ``ops`` callbacks for vhost lib + to work with the async copy devices: + + * ``transfer_data(vid, queue_id, descs, opaque_data, count)`` + + vhost invokes this function to submit copy data to the async devices. + For non-async_inorder capable devices, ``opaque_data`` could be used + for identifying the completed packets. + + * ``check_completed_copies(vid, queue_id, opaque_data, max_packets)`` + + vhost invokes this function to get the copy data completed by async + devices. + +* ``rte_vhost_async_channel_unregister(vid, queue_id)`` + + Unregister the async copy device channel from a vhost queue. + +* ``rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count)`` + + Submit an enqueue request to transmit ``count`` packets from host to + guest by async data path. Enqueue is not guaranteed to finish upon + the return of this API call. + + Applications must not free the packets submitted for enqueue until + the packets are completed. + +* ``rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count)`` + + Poll enqueue completion status from async data path. Completed + packets are returned to applications through ``pkts``. + Vhost-user Implementations --------------------------@@ -294,16 +362,16 @@ Guest memory requirement * Memory pre-allocation - For non-zerocopy, guest memory pre-allocation is not a must. This can help - save of memory. If users really want the guest memory to be pre-allocated - (e.g., for performance reason), we can add option ``-mem-prealloc`` when - starting QEMU. Or, we can lock all memory at vhost side which will force - memory to be allocated when mmap at vhost side; option --mlockall in - ovs-dpdk is an example in hand. + For non-zerocopy non-async data path, guest memory pre-allocation is + not a must. This can help save of memory. If users really want the + guest memory to be pre-allocated (e.g., for performance reason), we + can add option ``-mem-prealloc`` when starting QEMU. Or, we can lock + all memory at vhost side which will force memory to be allocated when + mmap at vhost side; option --mlockall in ovs-dpdk is an example in hand. - For zerocopy, we force the VM memory to be pre-allocated at vhost lib when - mapping the guest memory; and also we need to lock the memory to prevent - pages being swapped out to disk. + For async and zerocopy data path, we force the VM memory to be + pre-allocated at vhost lib when mapping the guest memory; and also we + need to lock the memory to prevent pages being swapped out to disk. * Memory sharing --
Quoted 1 line
2.18.4
Re: [dpdk-dev] [PATCH v2 1/2] doc: update guides for vhost async APIs
In reply to [email protected]Quoted 9 lines · On 7/22/20 5:01 PM, [email protected] wrote:
From: Patrick Fu <[email protected]>
Update vhost guides to document vhost async APIs
Signed-off-by Patrick Fu <[email protected]>--- doc/guides/prog_guide/vhost_lib.rst | 86 ++++++++++++++++++++++++++--- 1 file changed, 77 insertions(+), 9 deletions(-)
Thanks,
Maxime
PATCH v2 2/2 doc: update release notes for vhost async APIs
In reply to [email protected]From: Patrick Fu <[email protected]>
Update 20.08 release notes to include vhost async APIs
--- doc/guides/rel_notes/release_20_08.rst | 6 ++++++ 1 file changed, 6 insertions(+)
doc/guides/rel_notes/release_20_08.rst +6 -0
@@ -243,6 +243,12 @@ New Features of ingress packets to specific NIC queues. See the :doc:`../sample_app_ug/ipsec_secgw` for more details. +* **Added vhost async data path APIs.** + + 4 new APIs have been added to enable vhost async data path, including: + + * Async device channel register/unregister APIs + * Async packets enqueue/completion APIs (only split ring was implemented) Removed Items -------------
Signature · 1 line
2.18.4
Re: [dpdk-dev] [PATCH v2 2/2] doc: update release notes for vhost async APIs
Quoted 37 lines
-----Original Message-----
From: Fu, Patrick <[email protected]>
Sent: Wednesday, July 22, 2020 11:02 PM
To: [email protected]; [email protected]; Xia, Chenbo <[email protected]>
Cc: Fu, Patrick <[email protected]>
Subject: [PATCH v2 2/2] doc: update release notes for vhost async APIsFrom: Patrick Fu <[email protected]>
Update 20.08 release notes to include vhost async APIs
Signed-off-by Patrick Fu <[email protected]>--- doc/guides/rel_notes/release_20_08.rst | 6 ++++++ 1 file changed, 6 insertions(+)doc/guides/rel_notes/release_20_08.rst +6
-1@@ -243,6 +243,12 @@ New Features of ingress packets to specific NIC queues. See the :doc:`../sample_app_ug/ipsec_secgw` for more details. +* **Added vhost async data path APIs.** + + 4 new APIs have been added to enable vhost async data path, including: + + * Async device channel register/unregister APIs + * Async packets enqueue/completion APIs (only split ring was implemented) Removed Items ------------- --
Quoted 1 line
2.18.4
Re: [dpdk-dev] [PATCH v2 2/2] doc: update release notes for vhost async APIs
In reply to [email protected]Quoted 17 lines · On 7/22/20 5:01 PM, [email protected] wrote:
From: Patrick Fu <[email protected]>
Update 20.08 release notes to include vhost async APIs
Signed-off-by Patrick Fu <[email protected]>--- doc/guides/rel_notes/release_20_08.rst | 6 ++++++ 1 file changed, 6 insertions(+)doc/guides/rel_notes/release_20_08.rst +0
-0@@ -243,6 +243,12 @@ New Features of ingress packets to specific NIC queues. See the :doc:`../sample_app_ug/ipsec_secgw` for more details.
doc/guides/rel_notes/release_20_08.rst New Features · line 254
Quoted 10 lines
doc/guides/rel_notes/release_20_08.rst +6
-0+* **Added vhost async data path APIs.** + + 4 new APIs have been added to enable vhost async data path, including: + + * Async device channel register/unregister APIs + * Async packets enqueue/completion APIs (only split ring was implemented) Removed Items -------------
Thanks,
Maxime
Maxime Coquelin
AppliedRe: [dpdk-dev] [PATCH v2 0/2] update docs for vhost async APIs
In reply to [email protected]Quoted 11 lines · On 7/22/20 5:01 PM, [email protected] wrote:
From: Patrick Fu <[email protected]>
update docs for vhost async APIs
v2:
- fix typos invhost_lib.rst
- minor rewording invhost_lib.rst
- add more detailed API notes in release notes
- rebase the patch to 20.08 rc2
Quoted 8 lines
Patrick Fu (2):
doc: update guides for vhost async APIs
doc: update release notes for vhost async APIsdoc/guides/prog_guide/vhost_lib.rst | 86 +++++++++++++++++++++++--- doc/guides/rel_notes/release_20_08.rst | 6 ++ 2 files changed, 83 insertions(+), 9 deletions(-)
Applied to dpdk-next-virtio/master
Thanks!
Maxime