kni: add IOVA va support for kni
Re: [dpdk-dev] [PATCH v13 2/2] kni: support IOVA mode
In reply to Jerin JacobQuoted 9 lines · Jerin Jacob
Quoted 2 lines · David Marchand
So far, KNI could not work with IOVA as VA.
Your patchset adds support for IOVA as VA if kernel is >= 4.6.We need achive the following.
IOVA as PA has performance implication on KNI case. So we need to make IOVA as PA when KNI module is loaded.
- The current behaviour is:
* buses announce PA, nothing to do wrt KNI,
* buses announce VA or DC (whatever the considerations on iommu), if
physical addresses are available, then PA is used and KNI works,
- Now, with this new feature (on kernels >= 4.6), we can have KNI work with VA, the previous workaround can be skipped.
There is another consideration wrt performance, as a consequence, for kernels 4.6, if physical addresses are available, we can select PA for KNI.
_But_ I did not see people complaining about the current behavior. I see no reason to change this if the VA support can't be used (kernels < 4.6).
So how about (I inverted the #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 6, 0), it was causing me headaches):
lib/librte_eal/linux/eal/eal.c +0 -0
@@ -1073,6 +1073,11 @@ rte_eal_init(int argc, char **argv)*/ iova_mode =RTE_IOVA_VA ; RTE_LOG(DEBUG, EAL, "Physical
addresses are unavailable, selecting IOVA as VA mode.\n");
+#if defined(RTE_LIBRTE_KNI) && LINUX_VERSION_CODE >= KERNEL_VERSION(4, 6, 0)
+ } else if (rte_eal_check_module("rte_kni") == 1) {
+ iova_mode = RTE_IOVA_PA;
+ RTE_LOG(DEBUG, EAL, "Forcing IOVA as
'PA' for better perfomance with KNI\n");
+#endif
} else if (is_iommu_enabled()) {
/* we have an IOMMU, pick IOVA as VA mode */
iova_mode = RTE_IOVA_VA;
@@ -1085,7 +1090,7 @@ rte_eal_init(int argc, char **argv)
RTE_LOG(DEBUG, EAL, "IOMMU is not
available, selecting IOVA as PA mode.\n");
}
}-#ifdef RTE_LIBRTE_KNI
- /* Workaround for KNI which requires physical address to work */
+#if defined(RTE_LIBRTE_KNI) && LINUX_VERSION_CODE < KERNEL_VERSION(4, 6, 0)
+ /* Workaround for KNI which requires physical address
to work with kernels < 4.6 */
if (iova_mode == RTE_IOVA_VA &&
rte_eal_check_module("rte_kni") == 1) {Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v13 2/2] kni: support IOVA mode
Quoted 23 lines · David Marchand
Quoted 9 lines · Jerin Jacob
Quoted 2 lines · David Marchand
So far, KNI could not work with IOVA as VA.
Your patchset adds support for IOVA as VA if kernel is >= 4.6.We need achive the following.
IOVA as PA has performance implication on KNI case. So we need to make IOVA as PA when KNI module is loaded.
- The current behaviour is:
* buses announce PA, nothing to do wrt KNI,
* buses announce VA or DC (whatever the considerations on iommu), if
physical addresses are available, then PA is used and KNI works,- Now, with this new feature (on kernels >= 4.6), we can have KNI work with VA, the previous workaround can be skipped.
There is another consideration wrt performance, as a consequence, for kernels 4.6, if physical addresses are available, we can select PA for KNI.
Quoted 5 lines
_But_ I did not see people complaining about the current behavior. I see no reason to change this if the VA support can't be used (kernels < 4.6).
So how about (I inverted the #if
LINUX_VERSION_CODE>=KERNEL_VERSION(4, 6, 0), it was causing me headaches):
That works too David. Thanks for the review.
Quoted 36 lines
lib/librte_eal/linux/eal/eal.c +5
-0@@ -1073,6 +1073,11 @@ rte_eal_init(int argc, char **argv)*/ iova_mode =RTE_IOVA_VA ; RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n"); +#if defined(RTE_LIBRTE_KNI ) &&LINUX_VERSION_CODE >=KERNEL_VERSION (4 ,6 ,0 ) + } else if (rte_eal_check_module("rte_kni") == 1) { + iova_mode =RTE_IOVA_PA ; + RTE_LOG(DEBUG, EAL, "Forcing IOVA as 'PA' for better perfomance with KNI\n"); +#endif } else if (is_iommu_enabled()) {/* we have an IOMMU, pick IOVA as VA mode */ iova_mode = RTE_IOVA_VA;lib/librte_eal/linux/eal/eal.c +2
-2@@ -1085,7 +1090,7 @@ rte_eal_init(int argc, char **argv) RTE_LOG(DEBUG, EAL, "IOMMU is not available, selecting IOVA as PA mode.\n"); } } -#ifdef RTE_LIBRTE_KNI -/* Workaround for KNI which requires physical address to work */ +#if defined(RTE_LIBRTE_KNI ) &&LINUX_VERSION_CODE <KERNEL_VERSION (4 ,6 ,0 ) +/* Workaround for KNI which requires physical address to work with kernels < 4.6 */
if (iova_mode == RTE_IOVA_VA && rte_eal_check_module("rte_kni") == 1) {--
David Marchand
v14 cover kni: support IOVA mode
In reply to [email protected]From: Vamsi Attunuru <[email protected]>
---
V14 Changes:
* Simplified forcing IOVA as PA scheme.
* Updated documentation
V13 Changes:
* For KNI case, to restore the existing behaviour, forcing IOVA as PA if bus iommu returns IOVA_DC.
* Restored phys_addr and kni_lib defination checks.
* Simplified patch titles.
* Updated documentation about patch set's impact on performance and kernel version requirement.
V12 Changes:
* Removed previously added --legacy-kni eal option.
* Removed previously added kni specific mempool create routines and mempool populate routines.
V11 Changes:
* Added iova to kva address translation routines in kernel module to make it work in iova=va mode which enables DPDK to create kni devices on any kind of backed device/memory.
* Added ``--legacy-kni`` eal option to make existing KNI applications work with DPDK 19.11 and later versions.
* Removed previously added pci device info from kni device info struct.
V10 Changes:
* Fixed function return code on failure when min_chunk_size > pg_sz.
* Marked new mempool populate routine as EXPERIMENTAL.
V9 Changes:
* Used rte_mempool_ops_calc_mem_size() instead of default handler in the new mempool populate routine.
* Check min_chunk_size and return values.
* Removed ethdev_info memset to '0' and moved pci dev_info populate into kni_dev_pci_addr_get() routine.
* Addressed misc. review comments.
V8 Changes:
* Remove default mempool populate() routine changes.
* Add kni app specific mempool create & free routines.
* Add new mempool populate routine to allocate page-aligned memzones with page size to make sure all mempool objects reside on a page.
* Update release notes and map files.
V7 Changes:
* Removed previously proposed mempool flag and made those page boundary checks default in mempool populate() except for the objects size bigger than the size of page.
* Removed KNI example application related changes since pool related requirement is taken care in mempool lib.
* All PCI dev related info is moved under rte_eal_iova_mode() == VA check.
* Added wrapper functions in KNI module to hide IOVA checks and make address translation routines more readable.
* Updated IOVA mode checks that enforcing IOVA=PA mode when IOVA=VA mode is enabled.
V6 Changes:
* Added new mempool flag to ensure mbuf memory is not scattered across page boundaries.
* Added KNI kernel module required PCI device information.
* Modified KNI example application to create mempool with new mempool flag.
V5 changes:
* Fixed build issue with 32b build
V4 changes:
* Fixed build issues with older kernel versions
* This approach will only work with kernel above 4.4.0
David Marchand (1):
kni: support IOVA mode
Vamsi Attunuru (1):
kni: support IOVA mode in kernel module
doc/guides/prog_guide/kernel_nic_interface.rst | 14 +++++ doc/guides/rel_notes/release_19_11.rst | 13 ++++- kernel/linux/kni/compat.h | 14 +++++ kernel/linux/kni/kni_dev.h | 42 +++++++++++++++ kernel/linux/kni/kni_misc.c | 39 ++++++++++---- kernel/linux/kni/kni_net.c | 62 ++++++++++++++++++----- lib/librte_eal/linux/eal/eal.c | 11 +++- lib/librte_eal/linux/eal/include/rte_kni_common.h | 1 + lib/librte_kni/rte_kni.c | 5 ++ 9 files changed, 176 insertions(+), 25 deletions(-)
Signature · 1 line
2.8.4
PATCH v14 1/2 kni: support IOVA mode in kernel module
From: Vamsi Attunuru <[email protected]>
Patch adds support for kernel module to work in IOVA = VA mode by providing address translation routines to convert IOVA aka user space VA to kernel virtual addresses.
When compared with IOVA = PA mode, KNI netdev ports does not have any impact on performance with this approach, also there is no performance impact on IOVA = PA mode with this patch.
This approach does not work with the kernel versions less than 4.6.0.
--- kernel/linux/kni/compat.h | 14 +++++ kernel/linux/kni/kni_dev.h | 42 +++++++++++++++ kernel/linux/kni/kni_misc.c | 39 ++++++++++---- kernel/linux/kni/kni_net.c | 62 ++++++++++++++++++----- lib/librte_eal/linux/eal/include/rte_kni_common.h | 1 + 5 files changed, 136 insertions(+), 22 deletions(-)
kernel/linux/kni/compat.h +14 -0
@@ -121,3 +121,17 @@#if LINUX_VERSION_CODE >=KERNEL_VERSION (4 ,11 ,0 )#define HAVE_SIGNAL_FUNCTIONS_OWN_HEADER #endif + +#if KERNEL_VERSION (4 ,6 ,0 ) <=LINUX_VERSION_CODE + +#define HAVE_IOVA_TO_KVA_MAPPING_SUPPORT + +#if KERNEL_VERSION (4 ,9 ,0 ) >LINUX_VERSION_CODE +#define GET_USER_PAGES_REMOTE_API_V1 +#elif KERNEL_VERSION (4 ,9 ,0 ) ==LINUX_VERSION_CODE +#define GET_USER_PAGES_REMOTE_API_V2 +#else +#define GET_USER_PAGES_REMOTE_API_V3 +#endif + +#endif
kernel/linux/kni/kni_dev.h +42 -0
56 lines not on this page · raw message
kernel/linux/kni/kni_misc.c +30 -9
@@ -348,15 +348,36 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,strncpy (kni->name, dev_info.name,RTE_KNI_NAMESIZE );/* Translate user space info into kernel space info */ - kni->tx_q =phys_to_virt (dev_info.tx_phys); - kni->rx_q =phys_to_virt (dev_info.rx_phys); - kni->alloc_q =phys_to_virt (dev_info.alloc_phys); - kni->free_q =phys_to_virt (dev_info.free_phys); - - kni->req_q =phys_to_virt (dev_info.req_phys); - kni->resp_q =phys_to_virt (dev_info.resp_phys); - kni->sync_va = dev_info.sync_va; - kni->sync_kva =phys_to_virt (dev_info.sync_phys); +if (dev_info.iova_mode) { +#ifdef HAVE_IOVA_TO_KVA_MAPPING_SUPPORT + kni->tx_q =iova_to_kva (current, dev_info.tx_phys); + kni->rx_q =iova_to_kva (current, dev_info.rx_phys); + kni->alloc_q =iova_to_kva (current, dev_info.alloc_phys); + kni->free_q =iova_to_kva (current, dev_info.free_phys); + + kni->req_q =iova_to_kva (current, dev_info.req_phys); + kni->resp_q =iova_to_kva (current, dev_info.resp_phys); + kni->sync_va = dev_info.sync_va; + kni->sync_kva =iova_to_kva (current, dev_info.sync_phys); + kni->usr_tsk = current; + kni->iova_mode =1 ; +#else +pr_err ("KNI module does not support IOVA to VA translation\n" ); +return -EINVAL ; +#endif + }else { + + kni->tx_q =phys_to_virt (dev_info.tx_phys); + kni->rx_q =phys_to_virt (dev_info.rx_phys); + kni->alloc_q =phys_to_virt (dev_info.alloc_phys); + kni->free_q =phys_to_virt (dev_info.free_phys); + + kni->req_q =phys_to_virt (dev_info.req_phys); + kni->resp_q =phys_to_virt (dev_info.resp_phys); + kni->sync_va = dev_info.sync_va; + kni->sync_kva =phys_to_virt (dev_info.sync_phys); + kni->iova_mode =0 ; + } kni->mbuf_size = dev_info.mbuf_size;
kernel/linux/kni/kni_net.c +49 -13
119 lines not on this page · raw message
lib/librte_eal/linux/eal/include/rte_kni_common.h +1 -0
@@ -125,6 +125,7 @@ struct rte_kni_device_info {unsigned int min_mtu;unsigned int max_mtu;uint8_t mac_addr[6 ]; +uint8_t iova_mode; };#define KNI_DEVICE "kni"
Signature · 1 line
2.8.4
PATCH v14 2/2 kni: support IOVA mode
In reply to [email protected]From: David Marchand <[email protected]>
Current KNI implementation only operates in IOVA_PA mode patch adds required functionality to enable KNI in IOVA_VA mode.
Due to IOVA to KVA address translations, based on the KNI use case there can be a performance impact.
To mitigate the impact on performance,forcing IOVA to PA via eal "--iova-mode=pa" option can be used, IOVA_DC bus iommu scheme can also result in IOVA as PA.
During KNI creation, app's iova_mode details are passed to the KNI kernel module, accordingly kernel module translates PA/IOVA addresses to KVA and vice-versa.
--- doc/guides/prog_guide/kernel_nic_interface.rst | 14 ++++++++++++++ doc/guides/rel_notes/release_19_11.rst | 13 ++++++++++++- lib/librte_eal/linux/eal/eal.c | 11 +++++++++-- lib/librte_kni/rte_kni.c | 5 +++++ 4 files changed, 40 insertions(+), 3 deletions(-)
doc/guides/prog_guide/kernel_nic_interface.rst +14 -0
@@ -300,6 +300,20 @@ The sk_buff is then freed and the mbuf sent in the tx_q FIFO. The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_eth_tx_burst()``. It then puts the mbuf back in the cache. +IOVA = VA: Support +------------------ + +KNI operates in IOVA_VA scheme when + +- LINUX_VERSION_CODE >= KERNEL_VERSION(4, 6, 0) and +- eal option `iova-mode=va` is passed or bus IOVA scheme in the DPDK is selected + as RTE_IOVA_VA. + +Due to IOVA to KVA address translations, based on the KNI use case there +can be a performance impact. For mitigation, forcing IOVA to PA via eal +"--iova-mode=pa" option can be used, IOVA_DC bus iommu scheme can also +result in IOVA as PA. + Ethtool -------
doc/guides/rel_notes/release_19_11.rst +12 -1
@@ -292,8 +292,19 @@ New Features compilers store their internal representation of the source code that the linker uses at the final stage of compilation process. - See :doc:`../prog_guide/lto` for more information: +* **Added IOVA as VA support for KNI.** + + * Added IOVA = VA support for KNI, KNI can operate in IOVA = VA mode when + `iova-mode=va` eal option is passed to the application or when bus IOVA + scheme is selected as RTE_IOVA_VA. This mode only works on Linux Kernel + versions 4.6.0 and above. + * Due to IOVA to KVA address translations, based on the KNI use case there + can be a performance impact. For mitigation, forcing IOVA to PA via eal + "--iova-mode=pa" option can be used, IOVA_DC bus iommu scheme can also + result in IOVA as PA. + + See :doc:`../prog_guide/lto` for more information: Removed Items
lib/librte_eal/linux/eal/eal.c +9 -2
@@ -1073,6 +1073,11 @@ rte_eal_init(int argc, char **argv)*/ iova_mode =RTE_IOVA_VA ;RTE_LOG (DEBUG ,EAL ,"Physical addresses are unavailable, selecting IOVA as VA mode.\n" ); +#if defined(RTE_LIBRTE_KNI ) &&LINUX_VERSION_CODE >=KERNEL_VERSION (4 ,6 ,0 ) + } else if (rte_eal_check_module("rte_kni") == 1) { + iova_mode =RTE_IOVA_PA ; +RTE_LOG (DEBUG ,EAL ,"KNI is loaded, selecting IOVA as PA mode for better KNI perfomance.\n" ); +#endif }else if (is_iommu_enabled ()) {/* we have an IOMMU, pick IOVA as VA mode */ iova_mode =RTE_IOVA_VA ;
@@ -1085,8 +1090,10 @@ rte_eal_init(int argc, char **argv)RTE_LOG (DEBUG ,EAL ,"IOMMU is not available, selecting IOVA as PA mode.\n" ); } } -#ifdef RTE_LIBRTE_KNI -/* Workaround for KNI which requires physical address to work */ +#if defined(RTE_LIBRTE_KNI ) &&LINUX_VERSION_CODE <KERNEL_VERSION (4 ,6 ,0 ) +/* Workaround for KNI which requires physical address to work +* in kernels < 4.6 +*/ if (iova_mode ==RTE_IOVA_VA &&rte_eal_check_module ("rte_kni" ) ==1 ) {if (phys_addrs) {
lib/librte_kni/rte_kni.c +5 -0
@@ -10,6 +10,7 @@#include <fcntl.h> #include <unistd.h> #include <sys/ioctl.h> +#include <linux/version.h> #include <rte_spinlock.h> #include <rte_string_fns.h>
@@ -97,10 +98,12 @@ static volatile int kni_fd = -1;int rte_kni_init(unsigned int max_kni_ifaces __rte_unused) { +#if KERNEL_VERSION (4 ,6 ,0 ) >LINUX_VERSION_CODE if (rte_eal_iova_mode () !=RTE_IOVA_PA ) {RTE_LOG (ERR ,KNI ,"KNI requires IOVA as PA\n" );return -1 ; } +#endif /* Check FD and open */ if (kni_fd <0 ) {
@@ -302,6 +305,8 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni->group_id = conf->group_id; kni->mbuf_size = conf->mbuf_size; + dev_info.iova_mode = (rte_eal_iova_mode () ==RTE_IOVA_VA ) ?1 :0 ; + ret =ioctl (kni_fd,RTE_KNI_IOCTL_CREATE , &dev_info);if (ret <0 ) goto ioctl_fail;
Signature · 1 line
2.8.4
PATCH v9 2/5 kni: add IOVA=VA support in KNI lib
In reply to [email protected]From: Vamsi Attunuru <[email protected]>
Current KNI implementation only operates in IOVA=PA mode, patch adds required functionality in KNI lib to support IOVA=VA mode.
KNI kernel module requires device info to get iommu domain related information for IOVA addr related translations. Patch defines device related info in rte_kni_device_info structure and passes device info to the kernel KNI module when IOVA=VA mode is enabled.
--- lib/librte_eal/linux/eal/include/rte_kni_common.h | 8 ++++++ lib/librte_kni/Makefile | 1 + lib/librte_kni/meson.build | 1 + lib/librte_kni/rte_kni.c | 30 +++++++++++++++++++++++ 4 files changed, 40 insertions(+)
lib/librte_eal/linux/eal/include/rte_kni_common.h +8 -0
@@ -111,6 +111,13 @@ struct rte_kni_device_info {void * mbuf_va;phys_addr_t mbuf_phys; +/* PCI info */ +uint16_t vendor_id;/**< Vendor ID or PCI_ANY_ID. */ +uint16_t device_id;/**< Device ID or PCI_ANY_ID. */ +uint8_t bus;/**< Device bus */ +uint8_t devid;/**< Device ID */ +uint8_t function;/**< Device function. */ +uint16_t group_id;/**< Group ID */ uint32_t core_id;/**< core ID to bind for kernel thread */
@@ -121,6 +128,7 @@ struct rte_kni_device_info {unsigned mbuf_size;unsigned int mtu;uint8_t mac_addr[6 ]; +uint8_t iova_mode; };#define KNI_DEVICE "kni"
lib/librte_kni/Makefile +1 -0
@@ -7,6 +7,7 @@ include $(RTE_SDK)/mk/rte.vars.mk LIB = librte_kni.a CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing +CFLAGS += -I$(RTE_SDK)/drivers/bus/pci LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev EXPORT_MAP := rte_kni_version.map
lib/librte_kni/meson.build +1 -0
@@ -9,3 +9,4 @@ version = 2 sources = files('rte_kni.c') headers = files('rte_kni.h') deps += ['ethdev', 'pci'] +includes += include_directories('../../drivers/bus/pci')
lib/librte_kni/rte_kni.c +30 -0
@@ -14,6 +14,7 @@#include <rte_spinlock.h> #include <rte_string_fns.h> #include <rte_ethdev.h> +#include <rte_bus_pci.h> #include <rte_malloc.h> #include <rte_log.h> #include <rte_kni.h>
@@ -199,6 +200,27 @@ kni_release_mz(struct rte_kni *kni)rte_memzone_free (kni->m_sync_addr); } +static void +kni_dev_pci_addr_get (uint16_t port_id,struct rte_kni_device_info *kni_dev_info) +{ +const struct rte_pci_device *pci_dev; +struct rte_eth_dev_info dev_info; +const struct rte_bus *bus =NULL ; + +rte_eth_dev_info_get (port_id, &dev_info); + +if (dev_info.device) + bus =rte_bus_find_by_device (dev_info.device); +if (bus && !strcmp (bus->name,"pci" )) { + pci_dev =RTE_DEV_TO_PCI (dev_info.device); + kni_dev_info->bus = pci_dev->addr.bus; + kni_dev_info->devid = pci_dev->addr.devid; + kni_dev_info->function = pci_dev->addr.function; + kni_dev_info->vendor_id = pci_dev->id.vendor_id; + kni_dev_info->device_id = pci_dev->id.device_id; + } +} +struct rte_kni * rte_kni_alloc(struct rte_mempool *pktmbuf_pool, const struct rte_kni_conf *conf,
@@ -247,6 +269,12 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni->ops.port_id =UINT16_MAX ;memset (&dev_info,0 ,sizeof (dev_info)); + +if (rte_eal_iova_mode () ==RTE_IOVA_VA ) { +uint16_t port_id = conf->group_id; + +kni_dev_pci_addr_get (port_id, &dev_info); + } dev_info.core_id = conf->core_id; dev_info.force_bind = conf->force_bind; dev_info.group_id = conf->group_id;
@@ -300,6 +328,8 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni->group_id = conf->group_id; kni->mbuf_size = conf->mbuf_size; + dev_info.iova_mode = (rte_eal_iova_mode () ==RTE_IOVA_VA ) ?1 :0 ; + ret =ioctl (kni_fd,RTE_KNI_IOCTL_CREATE , &dev_info);if (ret <0 ) goto ioctl_fail;
Signature · 1 line
2.8.4
Re: [dpdk-dev] [PATCH v9 2/5] kni: add IOVA=VA support in KNI lib
Hi,
I believe iova_mode check should not be automatic and should be a part of rte_kni_conf.
What if I want to use KNI just as a pure virtual device without any connection to a real PCI device in an application that works in VA mode?
Best regards,
Igor
Quoted 132 lines · On Mon, Jul 29, 2019 at 3:14 PM <[email protected]> wrote:
From: Vamsi Attunuru <[email protected]>
Current KNI implementation only operates in IOVA=PA mode, patch adds required functionality in KNI lib to support IOVA=VA mode.
KNI kernel module requires device info to get iommu domain related information for IOVA addr related translations. Patch defines device related info in
rte_kni_device_infostructure and passes device info to the kernel KNI module when IOVA=VA mode is enabled.Signed-off-by Vamsi Attunuru <[email protected]>Signed-off-by Kiran Kumar K <[email protected]>--- lib/librte_eal/linux/eal/include/rte_kni_common.h | 8 ++++++ lib/librte_kni/Makefile | 1 + lib/librte_kni/meson.build | 1 + lib/librte_kni/rte_kni.c | 30 +++++++++++++++++++++++ 4 files changed, 40 insertions(+)lib/librte_eal/linux/eal/include/rte_kni_common.h +8
-0@@ -111,6 +111,13 @@ struct rte_kni_device_info {void * mbuf_va;phys_addr_t mbuf_phys; +/* PCI info */ +uint16_t vendor_id;/**< Vendor ID or PCI_ANY_ID. */ +uint16_t device_id;/**< Device ID or PCI_ANY_ID. */ +uint8_t bus;/**< Device bus */ +uint8_t devid;/**< Device ID */ +uint8_t function;/**< Device function. */ +uint16_t group_id;/**< Group ID */ uint32_t core_id;/**< core ID to bind for kernel thread */ @@ -121,6 +128,7 @@ struct rte_kni_device_info {unsigned mbuf_size;unsigned int mtu;uint8_t mac_addr[6 ]; +uint8_t iova_mode; };#define KNI_DEVICE "kni"lib/librte_kni/Makefile +1
-0@@ -7,6 +7,7 @@ include $(RTE_SDK)/mk/rte.vars.mk LIB = librte_kni.a CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing +CFLAGS += -I$(RTE_SDK)/drivers/bus/pci LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev EXPORT_MAP := rte_kni_version.maplib/librte_kni/meson.build +1
-0@@ -9,3 +9,4 @@ version = 2 sources = files('rte_kni.c') headers = files('rte_kni.h') deps += ['ethdev', 'pci'] +includes += include_directories('../../drivers/bus/pci')lib/librte_kni/rte_kni.c +30
-1@@ -14,6 +14,7 @@#include <rte_spinlock.h> #include <rte_string_fns.h> #include <rte_ethdev.h> +#include <rte_bus_pci.h> #include <rte_malloc.h> #include <rte_log.h> #include <rte_kni.h> @@ -199,6 +200,27 @@ kni_release_mz(struct rte_kni *kni)rte_memzone_free (kni->m_sync_addr); } +static void +kni_dev_pci_addr_get (uint16_t port_id,struct rte_kni_device_info *kni_dev_info) +{ +const struct rte_pci_device *pci_dev; +struct rte_eth_dev_info dev_info; +const struct rte_bus *bus =NULL ; + +rte_eth_dev_info_get (port_id, &dev_info); + +if (dev_info.device) + bus =rte_bus_find_by_device (dev_info.device); +if (bus && !strcmp (bus->name,"pci" )) { + pci_dev =RTE_DEV_TO_PCI (dev_info.device); + kni_dev_info->bus = pci_dev->addr.bus; + kni_dev_info->devid = pci_dev->addr.devid; + kni_dev_info->function = pci_dev->addr.function; + kni_dev_info->vendor_id = pci_dev->id.vendor_id; + kni_dev_info->device_id = pci_dev->id.device_id; + } +} +struct rte_kni * rte_kni_alloc(struct rte_mempool *pktmbuf_pool, const struct rte_kni_conf *conf,@@ -247,6 +269,12 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni->ops.port_id =UINT16_MAX ;memset (&dev_info,0 ,sizeof (dev_info)); + +if (rte_eal_iova_mode () ==RTE_IOVA_VA ) { +uint16_t port_id = conf->group_id; + +kni_dev_pci_addr_get (port_id, &dev_info); + } dev_info.core_id = conf->core_id; dev_info.force_bind = conf->force_bind; dev_info.group_id = conf->group_id;@@ -300,6 +328,8 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni->group_id = conf->group_id; kni->mbuf_size = conf->mbuf_size; + dev_info.iova_mode = (rte_eal_iova_mode () ==RTE_IOVA_VA ) ?1 :0 ; + ret =ioctl (kni_fd,RTE_KNI_IOCTL_CREATE , &dev_info);if (ret <0 ) goto ioctl_fail; --2.8.4
Re: [dpdk-dev] [EXT] Re: [PATCH v9 2/5] kni: add IOVA=VA support in KNI lib
Quoted 15 lines
-----Original Message-----
From: Igor Ryzhov <[email protected]>
Sent: Monday, July 29, 2019 5:55 PM
To: Vamsi Krishna Attunuru <[email protected]>
Cc: [email protected]; Thomas Monjalon <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Olivier Matz <[email protected]>; Ferruh Yigit <[email protected]>; [email protected]; Andrew Rybchenko <[email protected]>; Kiran Kumar Kokkilagadda <[email protected]>
Subject: [EXT] Re: [dpdk-dev] [PATCH v9 2/5] kni: add IOVA=VA support in KNI lib
Quoted 7 lines
External Email
----------------------------------------------------------------------
Hi,I believe
iova_modecheck should not be automatic and should be a part ofrte_kni_conf.
Changing the mode through rte_kni_conf will be too late, since already the environment would be set to VA/PA mode during the eal_init itself which can not be switched back based on the kni config.
Quoted 2 lines
What if I want to use KNI just as a pure virtual device without any connection to a real PCI device in an application that works in VA mode?
For the above use case, it will fail because there is no iommu domain present correspondingly, without that kernel kni module could not translate the memory that application(working in VA mode) provided to it.
One possible way is to use explicit iova=pa mode selection from eal command line option when there is vdev(not backed up by pci device) presents.
Other option will be adding a mechanism to find out the presence of vdev(without backed up by pci dev) before setting the environment's IOVA mode in eal_init(), accordingly change the mode to PA if both vdev and kni module are present.
@Ferruh, any other thoughts
BTW what is the plan for these KNI patches.
Quoted 138 lines
Best regards,
IgorQuoted 133 lines · On Mon, Jul 29, 2019 at 3:14 PM <[email protected]> wrote:
From: Vamsi Attunuru <[email protected]>
Current KNI implementation only operates in IOVA=PA mode, patch adds required functionality in KNI lib to support IOVA=VA mode.
KNI kernel module requires device info to get iommu domain related information for IOVA addr related translations. Patch defines device related info in
rte_kni_device_infostructure and passes device info to the kernel KNI module when IOVA=VA mode is enabled.Signed-off-by Vamsi Attunuru <[email protected]>Signed-off-by Kiran Kumar K <[email protected]>--- lib/librte_eal/linux/eal/include/rte_kni_common.h | 8 ++++++ lib/librte_kni/Makefile | 1 + lib/librte_kni/meson.build | 1 + lib/librte_kni/rte_kni.c | 30 +++++++++++++++++++++++ 4 files changed, 40 insertions(+)lib/librte_eal/linux/eal/include/rte_kni_common.h +8
-0@@ -111,6 +111,13 @@ struct rte_kni_device_info {void * mbuf_va;phys_addr_t mbuf_phys; +/* PCI info */ +uint16_t vendor_id;/**< Vendor ID or PCI_ANY_ID. */ +uint16_t device_id;/**< Device ID or PCI_ANY_ID. */ +uint8_t bus;/**< Device bus */ +uint8_t devid;/**< Device ID */ +uint8_t function;/**< Device function. */ +uint16_t group_id;/**< Group ID */ uint32_t core_id;/**< core ID to bind for kernel thread */ @@ -121,6 +128,7 @@ struct rte_kni_device_info {unsigned mbuf_size;unsigned int mtu;uint8_t mac_addr[6 ]; +uint8_t iova_mode; };#define KNI_DEVICE "kni"lib/librte_kni/Makefile index renamed from lib/librte_kni/Makefile +0
-0cbd6599..ab15d10 100644
lib/librte_kni/Makefile +1
-0@@ -7,6 +7,7 @@ include $(RTE_SDK)/mk/rte.vars.mk LIB = librte_kni.a CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing +CFLAGS += -I$(RTE_SDK)/drivers/bus/pci LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev EXPORT_MAP := rte_kni_version.maplib/librte_kni/meson.build +1
-0@@ -9,3 +9,4 @@ version = 2 sources = files('rte_kni.c') headers = files('rte_kni.h') deps += ['ethdev', 'pci'] +includes += include_directories('../../drivers/bus/pci')lib/librte_kni/rte_kni.c index renamed from lib/librte_kni/rte_kni.c +0
-04b51fb4..2aaaeaa 100644
lib/librte_kni/rte_kni.c +22
-0@@ -14,6 +14,7 @@#include <rte_spinlock.h> #include <rte_string_fns.h> #include <rte_ethdev.h> +#include <rte_bus_pci.h> #include <rte_malloc.h> #include <rte_log.h> #include <rte_kni.h> @@ -199,6 +200,27 @@ kni_release_mz(struct rte_kni *kni)rte_memzone_free (kni->m_sync_addr); } +static void +kni_dev_pci_addr_get (uint16_t port_id,struct rte_kni_device_info +*kni_dev_info) { +const struct rte_pci_device *pci_dev; +struct rte_eth_dev_info dev_info; +const struct rte_bus *bus =NULL ; + +rte_eth_dev_info_get (port_id, &dev_info); + +if (dev_info.device) + bus =rte_bus_find_by_device (dev_info.device); +if (bus && !strcmp (bus->name,"pci" )) { + pci_dev =RTE_DEV_TO_PCI (dev_info.device); + kni_dev_info->bus = pci_dev->addr.bus; + kni_dev_info->devid = pci_dev->addr.devid; + kni_dev_info->function = pci_dev->addr.function; + kni_dev_info->vendor_id = pci_dev->id.vendor_id; + kni_dev_info->device_id = pci_dev->id.device_id; + } +} +struct rte_kni * rte_kni_alloc(struct rte_mempool *pktmbuf_pool, const struct rte_kni_conf *conf, @@ -247,6 +269,12 @@rte_kni_alloc(struct
rte_mempool*pktmbuf_pool,kni->ops.port_id = UINT16_MAX; memset(&dev_info, 0, sizeof(dev_info));lib/librte_kni/rte_kni.c +6
-0+ +if (rte_eal_iova_mode () ==RTE_IOVA_VA ) { +uint16_t port_id = conf->group_id; + +kni_dev_pci_addr_get (port_id, &dev_info); + } dev_info.core_id = conf->core_id; dev_info.force_bind = conf->force_bind; dev_info.group_id = conf->group_id; @@ -300,6 +328,8 @@rte_kni_alloc(struct
rte_mempool*pktmbuf_pool,kni->group_id = conf->group_id; kni->mbuf_size = conf->mbuf_size;lib/librte_kni/rte_kni.c +3
-1+ dev_info.iova_mode = (rte_eal_iova_mode () ==RTE_IOVA_VA ) ?1 + :0 ; + ret =ioctl (kni_fd,RTE_KNI_IOCTL_CREATE , &dev_info);if (ret <0 ) goto ioctl_fail; --2.8.4
PATCH v9 3/5 kni: add app specific mempool create & free routine
In reply to [email protected]From: Vamsi Attunuru <[email protected]>
When KNI operates in IOVA = VA mode, it requires mbuf memory to be physically contiguous to ensure KNI kernel module could translate IOVA addresses properly. Patch adds a KNI specific mempool create routine to populate the KNI packet mbuf pool with memory objects that are being on a page.
KNI applications need to use this mempool create & free routines so that mbuf related requirements in IOVA = VA mode are handled inside those routines based on the enabled mode.
Updated the release notes with these new routine details.
--- doc/guides/rel_notes/release_19_08.rst | 6 ++++ examples/kni/main.c | 5 ++- lib/librte_kni/Makefile | 1 + lib/librte_kni/meson.build | 1 + lib/librte_kni/rte_kni.c | 60 ++++++++++++++++++++++++++++++++++ lib/librte_kni/rte_kni.h | 48 +++++++++++++++++++++++++++ lib/librte_kni/rte_kni_version.map | 2 ++ 7 files changed, 122 insertions(+), 1 deletion(-)
doc/guides/rel_notes/release_19_08.rst +6 -0
@@ -301,6 +301,12 @@ API Changes best-effort tc, and qsize field of struct ``rte_sched_port_params`` is changed to allow different size of the each queue. +* kni: ``rte_kni_pktmbuf_pool_create`` ``rte_kni_pktmbuf_pool_free`` functions + were introduced for KNI applications for creating & freeing packet pool. + Since IOVA=VA mode was added in KNI, packet pool's mbuf memory should be + physically contiguous for the KNI kernel module to work in IOVA=VA mode, + this requirement was taken care in the kni packet pool creation fucntions. + ABI Changes -----------
examples/kni/main.c +4 -1
@@ -975,7 +975,7 @@ main(int argc, char** argv)rte_exit (EXIT_FAILURE ,"Could not parse input parameters\n" );/* Create the mbuf pool */ - pktmbuf_pool =rte_pktmbuf_pool_create ("mbuf_pool" ,NB_MBUF , + pktmbuf_pool =rte_kni_pktmbuf_pool_create ("mbuf_pool" ,NB_MBUF ,MEMPOOL_CACHE_SZ ,0 ,MBUF_DATA_SZ ,rte_socket_id ());if (pktmbuf_pool ==NULL ) {rte_exit (EXIT_FAILURE ,"Could not initialise mbuf pool\n" );
@@ -1043,6 +1043,9 @@ main(int argc, char** argv)continue ;kni_free_kni (port); } + +rte_kni_pktmbuf_pool_free (pktmbuf_pool); +for (i =0 ; i <RTE_MAX_ETHPORTS ; i++)if (kni_port_params_array[i]) {rte_free (kni_port_params_array[i]);
lib/librte_kni/Makefile +1 -0
@@ -6,6 +6,7 @@ include $(RTE_SDK)/mk/rte.vars.mk # library name LIB = librte_kni.a +CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing CFLAGS += -I$(RTE_SDK)/drivers/bus/pci LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev
lib/librte_kni/meson.build +1 -0
@@ -1,6 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation +allow_experimental_apis = true if not is_linux or not dpdk_conf.get('RTE_ARCH_64') build = false reason = 'only supported on 64-bit linux'
lib/librte_kni/rte_kni.c +60 -0
72 lines not on this page · raw message
lib/librte_kni/rte_kni.h +48 -0
54 lines not on this page · raw message
lib/librte_kni/rte_kni_version.map +2 -0
@@ -20,4 +20,6 @@ EXPERIMENTAL { global: rte_kni_update_link; + rte_kni_pktmbuf_pool_create; + rte_kni_pktmbuf_pool_free; };
Signature · 1 line
2.8.4
PATCH v9 4/5 kni: add IOVA=VA support in KNI module
In reply to [email protected]From: Kiran Kumar K <[email protected]>
Patch adds support for kernel module to work in IOVA = VA mode, the idea is to get physical address from IOVA address using iommu_iova_to_phys API and later use phys_to_virt API to convert the physical address to kernel virtual address.
When compared with IOVA = PA mode, there is no performance drop with this approach.
This approach does not work with the kernel versions less than 4.4.0 because of API compatibility issues.
Patch also updates these support details in KNI documentation.
--- kernel/linux/kni/compat.h | 4 +++ kernel/linux/kni/kni_dev.h | 4 +++ kernel/linux/kni/kni_misc.c | 71 +++++++++++++++++++++++++++++++++++++++------ kernel/linux/kni/kni_net.c | 59 ++++++++++++++++++++++++++++--------- 4 files changed, 116 insertions(+), 22 deletions(-)
kernel/linux/kni/compat.h +4 -0
@@ -121,3 +121,7 @@#if LINUX_VERSION_CODE >=KERNEL_VERSION (4 ,11 ,0 )#define HAVE_SIGNAL_FUNCTIONS_OWN_HEADER #endif + +#if KERNEL_VERSION (4 ,4 ,0 ) <=LINUX_VERSION_CODE +#define HAVE_IOVA_AS_VA_SUPPORT +#endif
kernel/linux/kni/kni_dev.h +4 -0
@@ -25,6 +25,7 @@#include <linux/netdevice.h> #include <linux/spinlock.h> #include <linux/list.h> +#include <linux/iommu.h> #include <rte_kni_common.h> #define KNI_KTHREAD_RESCHEDULE_INTERVAL 5/* us */
@@ -41,6 +42,9 @@ struct kni_dev {/* kni list */ struct list_head list; +uint8_t iova_mode; +struct iommu_domain *domain; +uint32_t core_id;/* Core ID to bind */ char name[RTE_KNI_NAMESIZE ];/* Network device name */ struct task_struct *pthread;
kernel/linux/kni/kni_misc.c +62 -9
83 lines not on this page · raw message
kernel/linux/kni/kni_net.c +46 -13
116 lines not on this page · raw message
Signature · 1 line
2.8.4
PATCH v9 5/5 kni: modify IOVA mode checks to support VA
In reply to [email protected]From: Vamsi Attunuru <[email protected]>
Patch addresses checks in KNI and eal that enforce IOVA=PA when IOVA=VA mode is enabled, since KNI kernel module supports VA mode for kernel versions >= 4.4.0.
Updated KNI documentation with above details.
--- doc/guides/prog_guide/kernel_nic_interface.rst | 8 ++++++++ lib/librte_eal/linux/eal/eal.c | 4 +++- lib/librte_kni/rte_kni.c | 5 ----- 3 files changed, 11 insertions(+), 6 deletions(-)
doc/guides/prog_guide/kernel_nic_interface.rst +8 -0
@@ -291,6 +291,14 @@ The sk_buff is then freed and the mbuf sent in the tx_q FIFO. The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_eth_tx_burst()``. It then puts the mbuf back in the cache. +IOVA = VA: Support +------------------ + +KNI can be operated in IOVA as VA scheme when following criteria are fullfilled + +- LINUX_VERSION_CODE >= KERNEL_VERSION(4, 4, 0) +- eal param `--iova-mode va` is passed or bus IOVA scheme is set to RTE_IOVA_VA + Ethtool -------
lib/librte_eal/linux/eal/eal.c +3 -1
@@ -1068,12 +1068,14 @@ rte_eal_init(int argc, char **argv)/* Workaround for KNI which requires physical address to work */ if (iova_mode ==RTE_IOVA_VA &&rte_eal_check_module ("rte_kni" ) ==1 ) { +#if KERNEL_VERSION (4 ,4 ,0 ) >LINUX_VERSION_CODE if (phys_addrs) { iova_mode =RTE_IOVA_PA ; -RTE_LOG (WARNING ,EAL ,"Forcing IOVA as 'PA' because KNI module is loaded\n" ); +RTE_LOG (WARNING ,EAL ,"Forcing IOVA as 'PA' because KNI module does not support VA\n" ); }else {RTE_LOG (DEBUG ,EAL ,"KNI can not work since physical addresses are unavailable\n" ); } +#endif }#endif rte_eal_get_configuration ()->iova_mode = iova_mode;
lib/librte_kni/rte_kni.c +0 -5
@@ -99,11 +99,6 @@ static volatile int kni_fd = -1;int rte_kni_init(unsigned int max_kni_ifaces __rte_unused) { -if (rte_eal_iova_mode () !=RTE_IOVA_PA ) { -RTE_LOG (ERR ,KNI ,"KNI requires IOVA as PA\n" ); -return -1 ; - } -/* Check FD and open */ if (kni_fd <0 ) { kni_fd =open ("/dev/" KNI_DEVICE ,O_RDWR );
Signature · 1 line
2.8.4
Quoted 20 lines · On 22-Apr-19 5:39 AM, [email protected] wrote:
From: Kiran Kumar K <[email protected]>
With current KNI implementation kernel module will work only in IOVA=PA mode. This patch will add support for kernel module to work with IOVA=VA mode.
The idea is to get the physical address from iova address using api
iommu_iova_to_phys. Using this API, we will get the physical address from iova address and later usephys_to_virtAPI to convert the physical address to kernel virtual address.With this approach we have compared the performance with IOVA=PA and there is no difference observed. Seems like kernel is the overhead.
This approach will not work with the kernel versions less than 4.4.0 because of API compatibility issues.
Signed-off-by Kiran Kumar K <[email protected]>---
<snip>
Quoted 4 lines
kernel/linux/kni/kni_net.c +4
-0+/* iova to kernel virtual address */ +static void * +iova2kva (struct kni_dev *kni,void *pa) +{
kernel/linux/kni/kni_net.c static void kni_net_rx_normal(struct kni_dev *kni); · line 51
Quoted 10 lines
kernel/linux/kni/kni_net.c +10
-0+return phys_to_virt (iommu_iova_to_phys (kni->domain, + (dma_addr_t )pa)); +} + +static void * +iova2data_kva (struct kni_dev *kni,struct rte_kni_mbuf *m) +{ +return phys_to_virt ((iommu_iova_to_phys (kni->domain, + (dma_addr_t )m->buf_physaddr) + + m->data_off));
Does this account for mbufs crossing page boundary? In IOVA as VA mode, the mempool is likely allocated in one go, so the mempool allocator will not care for preventing mbufs from crossing page boundary. The data may very well start at the very end of a page, and continue through the beginning of next page, which will have a different physical address.
Signature · 2 lines
Thanks, Anatoly