introduce DMA memory mapping for external memory
cover introduce DMA memory mapping for external memory
This series is in continue to RFC[1].
The DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
The work in this patch moves the DMA mapping to vendor agnostic APIs. A new map and unmap ops were added to rte_bus structure. Implementation of those was done currently only on the PCI bus. The implementation takes the driver map and umap implementation as bypass to the VFIO mapping. That is, in case of no specific map/unmap from the PCI driver, VFIO mapping, if possible, will be used.
Application use with those APIs is quite simple:
* allocate memory
* take a device, and query its rte_device.
* call the bus map function for this device.
Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap APIs, leaving the PCI device APIs as the preferred option for the user.
[1] https://patches.dpdk.org/patch/47796/
Shahaf Shuler (6):
vfio: allow DMA map of memory for the default vfio fd
vfio: don't fail to DMA map if memory is already mapped
bus: introduce DMA memory mapping for external memory net/mlx5: refactor external memory registration net/mlx5: support PCI device DMA map and unmap
doc: deprecate VFIO DMA map APIs
doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 + drivers/bus/pci/pci_common.c | 78 +++++++ drivers/bus/pci/rte_bus_pci.h | 14 ++ drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 232 ++++++++++++++++--- drivers/net/mlx5/mlx5_rxtx.h | 5 + lib/librte_eal/common/eal_common_bus.c | 22 ++ lib/librte_eal/common/include/rte_bus.h | 57 +++++ lib/librte_eal/common/include/rte_vfio.h | 12 +- lib/librte_eal/linuxapp/eal/eal_vfio.c | 26 ++- lib/librte_eal/rte_eal_version.map | 2 + 12 files changed, 418 insertions(+), 38 deletions(-)
Signature · 1 line
2.12.0
PATCH 1/6 vfio: allow DMA map of memory for the default vfio fd
Enable users the option to call rte_vfio_dma_map with request to map to the default vfio fd.
--- lib/librte_eal/common/include/rte_vfio.h | 6 ++++-- lib/librte_eal/linuxapp/eal/eal_vfio.c | 14 ++++++++++++-- 2 files changed, 16 insertions(+), 4 deletions(-)
lib/librte_eal/common/include/rte_vfio.h +4 -2
@@ -347,7 +347,8 @@ rte_vfio_container_group_unbind(int container_fd, int iommu_group_num);* Perform DMA mapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. In case of -1 the default container +* fd will be used. * * @param vaddr * Starting virtual address of memory to be mapped.
@@ -370,7 +371,8 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr,* Perform DMA unmapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. In case of -1 the default container +* fd will be used. * * @param vaddr * Starting virtual address of memory to be unmapped.
lib/librte_eal/linuxapp/eal/eal_vfio.c +12 -2
@@ -1897,7 +1897,12 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd >0 ) { + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); + }else { +/* when no fd provided use the default. */ + vfio_cfg = &vfio_cfgs[0 ]; + }if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;
@@ -1917,7 +1922,12 @@ rte_vfio_container_dma_unmap(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd >0 ) { + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); + }else { +/* when no fd provided use the default. */ + vfio_cfg = &vfio_cfgs[0 ]; + }if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;
Signature · 1 line
2.12.0
Re: [PATCH 1/6] vfio: allow DMA map of memory for the default vfio fd
Hello Shahaf,
Quoted 14 lines · Shahaf Shuler
Enable users the option to call
rte_vfio_dma_mapwith request to map to the default vfio fd.Signed-off-by Shahaf Shuler <[email protected]>--- lib/librte_eal/common/include/rte_vfio.h | 6 ++++-- lib/librte_eal/linuxapp/eal/eal_vfio.c | 14 ++++++++++++-- 2 files changed, 16 insertions(+), 4 deletions(-)lib/librte_eal/common/include/rte_vfio.h +0
-0
lib/librte_eal/common/include/rte_vfio.h rte_vfio_container_group_unbind(int container_fd, int iommu_group_num); · line 351
Quoted 6 lines
lib/librte_eal/common/include/rte_vfio.h +2
-1@@ -347,7 +347,8 @@ rte_vfio_container_group_unbind(int container_fd, int iommu_group_num);* Perform DMA mapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. In case of -1 the default container +* fd will be used.
I think
#define RTE_VFIO_DEFAULT_CONTAINER_FD (-1)
might help reading the code that will later call these functions.
Quoted 18 lines
lib/librte_eal/common/include/rte_vfio.h +2
-1* * @param vaddr * Starting virtual address of memory to be mapped. @@ -370,7 +371,8 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr,* Perform DMA unmapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. In case of -1 the default container +* fd will be used. * * @param vaddr * Starting virtual address of memory to be unmapped. lib/librte_eal/linuxapp/eal/eal_vfio.c +0
-0
lib/librte_eal/linuxapp/eal/eal_vfio.c rte_vfio_container_dma_map(int container_fd, uint64_t vaddr, uint64_t iova, · line 1900
Quoted 5 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +1
-1@@ -1897,7 +1897,12 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd >0 ) {
Should it not be container_fd >= 0? This seems inconsistent with the doc above. Reading the code quickly, it's not clear that the container_fd==0 would be at vfio_cfgs[0], so this seems wrong.
lib/librte_eal/linuxapp/eal/eal_vfio.c rte_vfio_container_dma_map(int container_fd, uint64_t vaddr, uint64_t iova, · line 1905
Quoted 5 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +5
-0+ vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); + }else { +/* when no fd provided use the default. */ + vfio_cfg = &vfio_cfgs[0 ]; + }
Can you use:
vfio_cfg = default_vfio_cfg;
instead? Then the comment is redundant.
Actually, to keep with my comment above, it might be better to simply have
if (container_fd == RTE_VFIO_DEFAULT_CONTAINER_FD)
vfio_cfg = default_vfio_cfg;
else
vfio_cfg = get_vfio_cfg_by_group_num(container_fd);Quoted 20 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +6
-1if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;@@ -1917,7 +1922,12 @@ rte_vfio_container_dma_unmap(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd >0 ) { + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); + }else { +/* when no fd provided use the default. */ + vfio_cfg = &vfio_cfgs[0 ]; + }if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;Signature · 1 line
2.12.0
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH 1/6] vfio: allow DMA map of memory for the default vfio fd
Quoted 68 lines · Gaëtan Rivet
Hello Shahaf,
Quoted 14 lines · Shahaf Shuler
Enable users the option to call
rte_vfio_dma_mapwith request to map to the default vfio fd.Signed-off-by Shahaf Shuler <[email protected]>--- lib/librte_eal/common/include/rte_vfio.h | 6 ++++-- lib/librte_eal/linuxapp/eal/eal_vfio.c | 14 ++++++++++++-- 2 files changed, 16 insertions(+), 4 deletions(-)lib/librte_eal/common/include/rte_vfio.h +0
-0lib/librte_eal/common/include/rte_vfio.h rte_vfio_container_group_unbind(int container_fd, int iommu_group_num); · line 351
Quoted 6 lines
lib/librte_eal/common/include/rte_vfio.h +2
-1@@ -347,7 +347,8 @@ rte_vfio_container_group_unbind(int container_fd, int iommu_group_num);* Perform DMA mapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. In case of -1 the default container +* fd will be used. I think
#define
RTE_VFIO_DEFAULT_CONTAINER_FD(-1)might help reading the code that will later call these functions.
Quoted 18 lines
lib/librte_eal/common/include/rte_vfio.h +2
-1* * @param vaddr * Starting virtual address of memory to be mapped. @@ -370,7 +371,8 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr,* Perform DMA unmapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. In case of -1 the default container +* fd will be used. * * @param vaddr * Starting virtual address of memory to be unmapped. lib/librte_eal/linuxapp/eal/eal_vfio.c +0
-0lib/librte_eal/linuxapp/eal/eal_vfio.c rte_vfio_container_dma_map(int container_fd, uint64_t vaddr, uint64_t iova, · line 1900
Quoted 5 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +1
-1@@ -1897,7 +1897,12 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd >0 ) {Should it not be
container_fd>= 0? This seems inconsistent with the doc above. Reading the code quickly, it's not clear that the container_fd==0 would be at vfio_cfgs[0], so this seems wrong.lib/librte_eal/linuxapp/eal/eal_vfio.c rte_vfio_container_dma_map(int container_fd, uint64_t vaddr, uint64_t iova, · line 1905
Quoted 5 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +5
-0+ vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); + }else { +/* when no fd provided use the default. */ + vfio_cfg = &vfio_cfgs[0 ]; + }Can you use:
vfio_cfg=default_vfio_cfg;
Quoted 9 lines
instead? Then the comment is redundant.
Actually, to keep with my comment above, it might be better to simply haveif (container_fd == RTE_VFIO_DEFAULT_CONTAINER_FD) vfio_cfg = default_vfio_cfg; else vfio_cfg = get_vfio_cfg_by_group_num(container_fd);
Errm, copy error, this line should be
vfio_cfg = get_vfio_cfg_by_container_fd(container_fd);
of course.
Quoted 24 lines
Quoted 20 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +6
-1if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;@@ -1917,7 +1922,12 @@ rte_vfio_container_dma_unmap(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd >0 ) { + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); + }else { +/* when no fd provided use the default. */ + vfio_cfg = &vfio_cfgs[0 ]; + }if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;Signature · 1 line
2.12.0Signature · 2 lines
Gaëtan Rivet 6WIND
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH 1/6] vfio: allow DMA map of memory for the default vfio fd
In reply to Gaëtan RivetWednesday, February 13, 2019 11:45 AM, Gaëtan Rivet:
Quoted 4 lines
Subject: Re: [PATCH 1/6] vfio: allow DMA map of memory for the default vfio fd
Hello Shahaf,
Hi Gaetan.
Quoted 12 lines · Shahaf Shuler
Quoted 9 lines
Enable users the option to call
rte_vfio_dma_mapwith request to map to the default vfio fd.Signed-off-by Shahaf Shuler <[email protected]>--- lib/librte_eal/common/include/rte_vfio.h | 6 ++++-- lib/librte_eal/linuxapp/eal/eal_vfio.c | 14 ++++++++++++-- 2 files changed, 16 insertions(+), 4 deletions(-)
[...]
Quoted 4 lines
Can you use:
vfio_cfg=default_vfio_cfg;
Quoted 8 lines
instead? Then the comment is redundant.
Actually, to keep with my comment above, it might be better to simply haveif (container_fd == RTE_VFIO_DEFAULT_CONTAINER_FD) vfio_cfg = default_vfio_cfg; else vfio_cfg = get_vfio_cfg_by_group_num(container_fd);
Good suggestion. Will adjust in v2.
Quoted 25 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c rte_vfio_container_dma_unmap(int
Quoted 4 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +0
-0if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;@@ -1917,7 +1922,12 @@ rte_vfio_container_dma_unmap(int
container_fd,uint64_tvaddr,uint64_tiova,Quoted 14 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +6
-2return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd >0 ) { + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); + }else { +/* when no fd provided use the default. */ + vfio_cfg = &vfio_cfgs[0 ]; + }if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ; --Quoted 2 lines
2.12.0
--
Gaëtan Rivet
6WIND
Re: [PATCH 1/6] vfio: allow DMA map of memory for the default vfio fd
In reply to Shahaf ShulerQuoted 14 lines · Shahaf Shuler
Enable users the option to call
rte_vfio_dma_mapwith request to map to the default vfio fd.Signed-off-by Shahaf Shuler <[email protected]>--- lib/librte_eal/common/include/rte_vfio.h | 6 ++++-- lib/librte_eal/linuxapp/eal/eal_vfio.c | 14 ++++++++++++-- 2 files changed, 16 insertions(+), 4 deletions(-)lib/librte_eal/common/include/rte_vfio.h +0
-0
lib/librte_eal/common/include/rte_vfio.h rte_vfio_container_group_unbind(int container_fd, int iommu_group_num); · line 351
Quoted 6 lines
lib/librte_eal/common/include/rte_vfio.h +2
-1@@ -347,7 +347,8 @@ rte_vfio_container_group_unbind(int container_fd, int iommu_group_num);* Perform DMA mapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. In case of -1 the default container +* fd will be used.
I haven't looked at the patchset in depth yet, however, this changes the public API, so warrants a note in release notes.
Signature · 2 lines
Thanks, Anatoly
PATCH 2/6 vfio: don't fail to DMA map if memory is already mapped
In reply to Shahaf ShulerCurrently vfio DMA map function will fail in case the same memory segment is mapped twice.
This is too strict, as this is not an error to map the same memory twice.
Instead, use the kernel return value to detect such state and have the DMA function to return as successful.
For type1 mapping the kernel driver first implementation returns EBUSY and since kernel 3.11 returns EEXISTS. For spapr mapping EBUSY is returned since kernel 4.10.
--- lib/librte_eal/linuxapp/eal/eal_vfio.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_vfio.c +10 -2
@@ -1263,7 +1263,11 @@ vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,VFIO_DMA_MAP_FLAG_WRITE ; ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map); -if (ret) { +/** +* In case the mapping was already done EEXIST will be +* returned from kernel. +*/ +if ((ret != -EEXIST ) && (ret != -EBUSY )) {RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , errno,strerror (errno));return -1 ;
@@ -1324,7 +1328,11 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,VFIO_DMA_MAP_FLAG_WRITE ; ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map); -if (ret) { +/** +* In case the mapping was already done EBUSY will be +* returned from kernel. +*/ +if (ret != -EBUSY ) {RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , errno,strerror (errno));return -1 ;
Signature · 1 line
2.12.0
Re: [PATCH 2/6] vfio: don't fail to DMA map if memory is already mapped
Quoted 6 lines · Shahaf Shuler
Currently vfio DMA map function will fail in case the same memory segment is mapped twice.
This is too strict, as this is not an error to map the same memory twice.
Quoted 7 lines
Instead, use the kernel return value to detect such state and have the DMA function to return as successful.
For type1 mapping the kernel driver first implementation returns EBUSY and since kernel 3.11 returns EEXISTS. For spapr mapping EBUSY is returned since kernel 4.10.
What is the earliest version supported by DPDK? I thought 3.10 was dropped, should we care about the 3.11 return value?
Quoted 10 lines
Signed-off-by Shahaf Shuler <[email protected]>--- lib/librte_eal/linuxapp/eal/eal_vfio.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal_vfio.c +0
-0
lib/librte_eal/linuxapp/eal/eal_vfio.c vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova, · line 1270
Quoted 9 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +5
-1@@ -1263,7 +1263,11 @@ vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,VFIO_DMA_MAP_FLAG_WRITE ; ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map); -if (ret) { +/** +* In case the mapping was already done EEXIST will be +* returned from kernel. +*/ +if ((ret != -EEXIST ) && (ret != -EBUSY )) {
Won't a ret == 0 trigger the error then?
It seems ifdef about linux versions are not common in vfio code, but bar that I think it would be cleaner to restrict the acceptable error to it.
When a version will be dropped it will be much easier to remove the related code instead of digging in the commit logs, and leaving both would not be clean.
Quoted 19 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +5
-1RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , errno,strerror (errno));return -1 ;@@ -1324,7 +1328,11 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,VFIO_DMA_MAP_FLAG_WRITE ; ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map); -if (ret) { +/** +* In case the mapping was already done EBUSY will be +* returned from kernel. +*/ +if (ret != -EBUSY ) {RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , errno,strerror (errno));return -1 ;Signature · 1 line
2.12.0
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH 2/6] vfio: don't fail to DMA map if memory is already mapped
Wednesday, February 13, 2019 11:59 AM, Gaëtan RiveT:
Quoted 13 lines
Subject: Re: [PATCH 2/6] vfio: don't fail to DMA map if memory is already mapped
Quoted 9 lines · Shahaf Shuler
Currently vfio DMA map function will fail in case the same memory segment is mapped twice.
This is too strict, as this is not an error to map the same memory twice.
Instead, use the kernel return value to detect such state and have the DMA function to return as successful.
Quoted 7 lines
Quoted 4 lines
For type1 mapping the kernel driver first implementation returns EBUSY and since kernel 3.11 returns EEXISTS. For spapr mapping EBUSY is returned since kernel 4.10.
What is the earliest version supported by DPDK? I thought 3.10 was dropped, should we care about the 3.11 return value?
According to DPDK doc it is 3.16, however compatibility for Centos/RH 7 should be kept. I have looked on Centos 7 source code (linux-3.10.0-957.5.1.el7), it uses EEXIST for the vfio type 1.
So I think, I can drop the EBUSY error check and leave only the EEXIST one.
Quoted 17 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c vfio_type1_dma_mem_map(int
Quoted 12 lines
Signed-off-by Shahaf Shuler <[email protected]>--- lib/librte_eal/linuxapp/eal/eal_vfio.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal_vfio.c +0
-0@@ -1263,7 +1263,11 @@ vfio_type1_dma_mem_map(int
vfio_container_fd,uint64_tvaddr,uint64_tiova,lib/librte_eal/linuxapp/eal/eal_vfio.c vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,
Quoted 3 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +0
-0VFIO_DMA_MAP_FLAG_WRITE ; ret = ioctl(vfio_container_fd, VFIO_IOMMU_MAP_DMA,
&dma_map);
Quoted 8 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova, · line 1270
Quoted 6 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +5
-1-if (ret) { +/** +* In case the mapping was already done EEXIST will be +* returned from kernel. +*/ +if ((ret != -EEXIST ) && (ret != -EBUSY )) {Won't a ret == 0 trigger the error then?
Yes good catch.
Quoted 7 lines
It seems ifdef about linux versions are not common in vfio code, but bar that I think it would be cleaner to restrict the acceptable error to it.
When a version will be dropped it will be much easier to remove the related code instead of digging in the commit logs, and leaving both would not be clean.
Relaying on kernel version is not safe enough. Many distro backport to their kernel w/o updating its version. This will lead to unexpected behavior.
Quoted 28 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c vfio_type1_dma_mem_map(int · line 1271
Quoted 2 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +0
-0RTE_LOG(ERR, EAL, " cannot set up DMA remapping,error %i (%s)\n",
lib/librte_eal/linuxapp/eal/eal_vfio.c vfio_spapr_dma_do_map(int vfio_container_fd,
Quoted 3 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +0
-0errno, strerror(errno));return -1 ;@@ -1324,7 +1328,11 @@ vfio_spapr_dma_do_map(int vfio_container_fd,
uint64_tvaddr,uint64_tiova,lib/librte_eal/linuxapp/eal/eal_vfio.c vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,
Quoted 3 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +0
-0VFIO_DMA_MAP_FLAG_WRITE ; ret = ioctl(vfio_container_fd, VFIO_IOMMU_MAP_DMA,
&dma_map);lib/librte_eal/linuxapp/eal/eal_vfio.c vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,
Quoted 7 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +5
-1-if (ret) { +/** +* In case the mapping was already done EBUSY will be +* returned from kernel. +*/ +if (ret != -EBUSY ) { RTE_LOG(ERR, EAL, " cannot set up DMA remapping,error %i (%s)\n",
Quoted 5 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +0
-1errno, strerror(errno));return -1 ; --2.12.0
--
Gaëtan Rivet
6WIND
PATCH 3/6 bus: introduce DMA memory mapping for external memory
In reply to Shahaf ShulerThe DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
The work in this patch moves the DMA mapping to vendor agnostic APIs. A new map and unmap ops were added to rte_bus structure. Implementation of those was done currently only on the PCI bus. The implementation takes the driver map and umap implementation as bypass to the VFIO mapping. That is, in case of no specific map/unmap from the PCI driver, VFIO mapping, if possible, will be used.
Application use with those APIs is quite simple:
* allocate memory
* take a device, and query its rte_device.
* call the bus map function for this device.
Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap APIs, leaving the PCI device APIs as the preferred option for the user.
--- drivers/bus/pci/pci_common.c | 78 ++++++++++++++++++++++++++++ drivers/bus/pci/rte_bus_pci.h | 14 +++++ lib/librte_eal/common/eal_common_bus.c | 22 ++++++++ lib/librte_eal/common/include/rte_bus.h | 57 ++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 2 + 5 files changed, 173 insertions(+)
drivers/bus/pci/pci_common.c +78 -0
@@ -528,6 +528,82 @@ pci_unplug(struct rte_device *dev)return ret; } +/** +* DMA Map memory segment to device. After a successful call the device +* will be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be mapped. +* @param iova +* Starting IOVA address of memory to be mapped. +* @param len +* Length of memory segment being mapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/ +static int __rte_experimental +pci_dma_map (struct rte_device *dev,void *addr,uint64_t iova,size_t len) +{ +struct rte_pci_device *pdev =RTE_DEV_TO_PCI (dev); + +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ; +return -rte_errno; + } +if (pdev->driver->map) +return pdev->driver->map (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_map (-1 , (uintptr_t )addr, iova, + len); + rte_errno =ENOTSUP ; +return -rte_errno; +} + +/** +* Un-map memory segment to device. After a successful call the device +* will not be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be unmapped. +* @param iova +* Starting IOVA address of memory to be unmapped. +* @param len +* Length of memory segment being unmapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/ +static int __rte_experimental +pci_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova,size_t len) +{ +struct rte_pci_device *pdev =RTE_DEV_TO_PCI (dev); + +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ; +return -rte_errno; + } +if (pdev->driver->unmap) +return pdev->driver->unmap (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_unmap (-1 , (uintptr_t )addr, iova, + len); + rte_errno =ENOTSUP ; +return -rte_errno; +} +struct rte_pci_bus rte_pci_bus = { .bus = { .scan = rte_pci_scan,
@@ -536,6 +612,8 @@ struct rte_pci_bus rte_pci_bus = { .plug = pci_plug, .unplug = pci_unplug, .parse = pci_parse, + .map = pci_dma_map, + .unmap = pci_dma_unmap, .get_iommu_class = rte_pci_get_iommu_class, .dev_iterate = rte_pci_dev_iterate, .hot_unplug_handler = pci_hot_unplug_handler,
drivers/bus/pci/rte_bus_pci.h +14 -0
@@ -114,6 +114,18 @@ typedef int (pci_probe_t)(struct rte_pci_driver *, struct rte_pci_device *);typedef int (pci_remove_t )(struct rte_pci_device *);/** +* Driver-specific DMA mapping. +*/ +typedef int (pci_dma_map_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** +* Driver-specific DMA unmapping. +*/ +typedef int (pci_dma_unmap_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver {
@@ -122,6 +134,8 @@ struct rte_pci_driver {struct rte_pci_bus *bus;/**< PCI bus reference. */ pci_probe_t *probe;/**< Device Probe function. */ pci_remove_t *remove;/**< Device Remove function. */ +pci_dma_map_t *map;/**< device dma map function. */ +pci_dma_unmap_t *unmap;/**< device dma unmap function. */ const struct rte_pci_id *id_table;/**< ID table, NULL terminated. */ uint32_t drv_flags;/**< Flags RTE_PCI_DRV_*. */ };
lib/librte_eal/common/eal_common_bus.c +22 -0
@@ -285,3 +285,25 @@ rte_bus_sigbus_handler(const void *failure_addr)return ret; } + +int __rte_experimental +rte_bus_dma_map (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->map ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -rte_errno; + } +return dev->bus->map (dev, addr, iova, len); +} + +int __rte_experimental +rte_bus_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->unmap ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -rte_errno; + } +return dev->bus->unmap (dev, addr, iova, len); +}
lib/librte_eal/common/include/rte_bus.h +57 -0
75 lines not on this page · raw message
lib/librte_eal/rte_eal_version.map +2 -0
@@ -364,4 +364,6 @@ EXPERIMENTAL { rte_service_may_be_active; rte_socket_count; rte_socket_id_by_idx; + rte_bus_dma_map; + rte_bus_dma_unmap; };
Signature · 1 line
2.12.0
Re: [PATCH 3/6] bus: introduce DMA memory mapping for external memory
Quoted 17 lines · Shahaf Shuler
The DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
Quoted 6 lines
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
How are those other vendors' devices mapped initially right now? Are they using #2 scheme instead? Then the user will remap everything using #3?
Would it be interesting to be able to describe a mapping prior to probing a device and refer to it upon hotplug?
Quoted 6 lines
The work in this patch moves the DMA mapping to vendor agnostic APIs. A new map and unmap ops were added to
rte_busstructure. Implementation of those was done currently only on the PCI bus. The implementation takes the driver map and umap implementation as bypass to the VFIO mapping. That is, in case of no specific map/unmap from the PCI driver, VFIO mapping, if possible, will be used.
This paragraph should be rewritten to better fit a commit log.
Quoted 5 lines
Application use with those APIs is quite simple:
* allocate memory
* take a device, and query itsrte_device.
* call the bus map function for this device.
Is the device already configured with the existing mappings? Should the application stop it before attempting to map its allocated memory?
Quoted 27 lines
Future work will deprecate the
rte_vfio_dma_mapandrte_vfio_dma_unmapAPIs, leaving the PCI device APIs as the preferred option for the user.Signed-off-by Shahaf Shuler <[email protected]>--- drivers/bus/pci/pci_common.c | 78 ++++++++++++++++++++++++++++ drivers/bus/pci/rte_bus_pci.h | 14 +++++ lib/librte_eal/common/eal_common_bus.c | 22 ++++++++ lib/librte_eal/common/include/rte_bus.h | 57 ++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 2 + 5 files changed, 173 insertions(+)drivers/bus/pci/pci_common.c +6
-0@@ -528,6 +528,82 @@ pci_unplug(struct rte_device *dev)return ret; } +/** +* DMA Map memory segment to device. After a successful call the device +* will be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device.
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 546
Quoted 10 lines
drivers/bus/pci/pci_common.c +10
-0+* @param addr +* Starting virtual address of memory to be mapped. +* @param iova +* Starting IOVA address of memory to be mapped. +* @param len +* Length of memory segment being mapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/
This doc should be on the callback typedef, not their implementation. The rte_errno error spec should also be documented higher-up in the abstraction pile, on the bus callback I think. Everyone should follow the same error codes for applications to really be able to use any implementation generically.
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 547
Quoted 1 line
drivers/bus/pci/pci_common.c +1
-0+static int __rte_experimental
The __rte_experimental is not necessary in compilation units themselves, only in the headers.
In any case, it would only be the publicly available API that must be marked as such, so more the callback typedefs than their implementations.
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 552
Quoted 5 lines
drivers/bus/pci/pci_common.c +5
-0+pci_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len) +{ + struct rte_pci_device *pdev = RTE_DEV_TO_PCI(dev); + +if (!pdev || !pdev->driver) {
pdev cannot be null here, nor should its driver be.
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 564
Quoted 12 lines
drivers/bus/pci/pci_common.c +12
-0+ rte_errno =EINVAL ; +return -rte_errno; + } +if (pdev->driver->map) +return pdev->driver->map (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_map (-1 , (uintptr_t )addr, iova, + len);
Reiterating: RTE_VFIO_DEFAULT_CONTAINER_FD is more readable I think than -1 here.
Quoted 11 lines
drivers/bus/pci/pci_common.c +11
-0+ rte_errno =ENOTSUP ; +return -rte_errno; +} + +/** +* Un-map memory segment to device. After a successful call the device +* will not be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 585
Quoted 10 lines
drivers/bus/pci/pci_common.c +10
-0+* Starting virtual address of memory to be unmapped. +* @param iova +* Starting IOVA address of memory to be unmapped. +* @param len +* Length of memory segment being unmapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/ +static int __rte_experimental
Same as before for __rte_experimental and doc.
Quoted 57 lines
drivers/bus/pci/pci_common.c +23
-0+pci_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, size_t len) +{ + struct rte_pci_device *pdev = RTE_DEV_TO_PCI(dev); + +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ; +return -rte_errno; + } +if (pdev->driver->unmap) +return pdev->driver->unmap (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_unmap (-1 , (uintptr_t )addr, iova, + len); + rte_errno =ENOTSUP ; +return -rte_errno; +} +struct rte_pci_bus rte_pci_bus = { .bus = { .scan = rte_pci_scan,@@ -536,6 +612,8 @@ struct rte_pci_bus rte_pci_bus = { .plug = pci_plug, .unplug = pci_unplug, .parse = pci_parse, + .map = pci_dma_map, + .unmap = pci_dma_unmap, .get_iommu_class = rte_pci_get_iommu_class, .dev_iterate = rte_pci_dev_iterate, .hot_unplug_handler = pci_hot_unplug_handler,drivers/bus/pci/rte_bus_pci.h +12
-0@@ -114,6 +114,18 @@ typedef int (pci_probe_t)(struct rte_pci_driver *, struct rte_pci_device *);typedef int (pci_remove_t )(struct rte_pci_device *);/** +* Driver-specific DMA mapping. +*/ +typedef int (pci_dma_map_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** +* Driver-specific DMA unmapping. +*/ +typedef int (pci_dma_unmap_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver {
drivers/bus/pci/rte_bus_pci.h struct rte_pci_driver { · line 138
Quoted 5 lines
drivers/bus/pci/rte_bus_pci.h +2
-0@@ -122,6 +134,8 @@ struct rte_pci_driver {struct rte_pci_bus *bus;/**< PCI bus reference. */ pci_probe_t *probe;/**< Device Probe function. */ pci_remove_t *remove;/**< Device Remove function. */ +pci_dma_map_t *map;/**< device dma map function. */ +pci_dma_unmap_t *unmap;/**< device dma unmap function. */
I'd call both callbacks dma_map and dma_unmap. It's clearer and more consistent.
Quoted 23 lines
drivers/bus/pci/rte_bus_pci.h +0
-0const struct rte_pci_id *id_table;/**< ID table, NULL terminated. */ uint32_t drv_flags;/**< Flags RTE_PCI_DRV_*. */ };lib/librte_eal/common/eal_common_bus.c +12
-0@@ -285,3 +285,25 @@ rte_bus_sigbus_handler(const void *failure_addr)return ret; } + +int __rte_experimental +rte_bus_dma_map (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->map ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -rte_errno; + } +return dev->bus->map (dev, addr, iova, len); +} +
lib/librte_eal/common/eal_common_bus.c rte_bus_sigbus_handler(const void *failure_addr) · line 309
Quoted 10 lines
lib/librte_eal/common/eal_common_bus.c +10
-0+int __rte_experimental +rte_bus_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->unmap ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -rte_errno; + } +return dev->bus->unmap (dev, addr, iova, len); +}
These functions should be called rte_dev_dma_{map,unmap} and be part of eal_common_dev.c instead.
Quoted 54 lines
lib/librte_eal/common/include/rte_bus.h +42
-0@@ -168,6 +168,48 @@ typedef int (*rte_bus_unplug_t)(struct rte_device *dev);typedef int (*rte_bus_parse_t )(const char *name,void *addr);/** +* Bus specific DMA map function. +* After a successful call, the memory segment will be mapped to the +* given device. +* +* @param dev +* Device pointer. +* @param addr +* Virtual address to map. +* @param iova +* IOVA address to map. +* @param len +* Length of the memory segment being mapped. +* +* @return +* 0 if mapping was successful. +* Negative value and rte_errno is set otherwise. +*/ +typedef int (*rte_bus_map_t )(struct rte_device *dev,void *addr, +uint64_t iova,size_t len); + +/** +* Bus specific DMA unmap function. +* After a successful call, the memory segment will no longer be +* accessible by the given device. +* +* @param dev +* Device pointer. +* @param addr +* Virtual address to unmap. +* @param iova +* IOVA address to unmap. +* @param len +* Length of the memory segment being mapped. +* +* @return +* 0 if un-mapping was successful. +* Negative value and rte_errno is set otherwise. +*/ +typedef int (*rte_bus_unmap_t )(struct rte_device *dev,void *addr, +uint64_t iova,size_t len); + +/** * Implement a specific hot-unplug handler, which is responsible for * handle the failure when device be hot-unplugged. When the event of * hot-unplug be detected, it could call this function to handle
lib/librte_eal/common/include/rte_bus.h struct rte_bus { · line 284
Quoted 5 lines
lib/librte_eal/common/include/rte_bus.h +2
-0@@ -238,6 +280,8 @@ struct rte_bus {rte_bus_plug_t plug;/**< Probe single device for drivers */ rte_bus_unplug_t unplug;/**< Remove single device from driver */ rte_bus_parse_t parse;/**< Parse a device name */ +rte_bus_map_t map;/**< DMA map for device in the bus */ +rte_bus_unmap_t unmap;/**< DMA unmap for device in the bus */
Same as for the driver callbacks, dma_map and dma_unmap seem a better fit for the field names.
Quoted 10 lines
lib/librte_eal/common/include/rte_bus.h +3
-0struct rte_bus_conf conf;/**< Bus configuration */ rte_bus_get_iommu_class_t get_iommu_class;/**< Get iommu class */ rte_dev_iterate_t dev_iterate;/**< Device iterator. */ @@ -356,6 +400,19 @@ struct rte_bus *rte_bus_find_by_name(const char *busname);enum rte_iova_mode rte_bus_get_iommu_class (void );/** +* Wrapper to call the bus specific DMA map function. +*/ +int __rte_experimental
lib/librte_eal/common/include/rte_bus.h struct rte_bus *rte_bus_find_by_name(const char *busname); · line 415
Quoted 10 lines
lib/librte_eal/common/include/rte_bus.h +10
-0+rte_bus_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len); + +/** +* Wrapper to call the bus specific DMA unmap function. +*/ +int __rte_experimental +rte_bus_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova, +size_t len); + +/**
Same as earlier -> these seem device-level functions, not bus-related. You won't map those addresses to all devices on the bus.
Quoted 17 lines
lib/librte_eal/common/include/rte_bus.h +0
-0* Helper for Bus registration. * The constructor has higher priority than PMD constructors. */ lib/librte_eal/rte_eal_version.map +2
-0@@ -364,4 +364,6 @@ EXPERIMENTAL { rte_service_may_be_active; rte_socket_count; rte_socket_id_by_idx; + rte_bus_dma_map; + rte_bus_dma_unmap; };Signature · 1 line
2.12.0
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH 3/6] bus: introduce DMA memory mapping for external memory
Wednesday, February 13, 2019 1:17 PM, Gaëtan Rivet:
Quoted 26 lines
Subject: Re: [PATCH 3/6] bus: introduce DMA memory mapping for external memory
Quoted 1 line · Shahaf Shuler
The DPDK APIs expose 3 different modes to work with memory used for
DMA:
Quoted 2 lines
1. Use the DPDK owned memory (backed by the DPDK provided
hugepages).
Quoted 2 lines
This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the
DPDK
Quoted 14 lines
layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Quoted 7 lines
Quoted 4 lines
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).How are those other vendors' devices mapped initially right now? Are they using #2 scheme instead? Then the user will remap everything using #3?
It is not a re-map, it is a completely different mode for the memory management.
The first question to ask is "how the application wants to manage its memory" ?
If it is either #1 or #2 above, no problem to make the mapping internal on the "other vendor devices" as they can register to the memory event callback which trigger every time new memory is added to the DPDK memory management system. For #3 the memory does not exists in the DPDK memory management system, and no memory events. Hence the application needs to explicitly call the dma MAP.
The change on this patch is just to make it more generic than calling only VFIO.
Quoted 3 lines
Would it be interesting to be able to describe a mapping prior to probing a device and refer to it upon hotplug?
Not sure it is an interesting use case. I don't see the need to setup the application memory before the probing of the devices.
Regarding hotplug - this is a feature we can add on top of this series (for example if device was removed and hotplug back). This will require to store the mapping on some database, like VFIO does.
Quoted 12 lines
Quoted 5 lines
The work in this patch moves the DMA mapping to vendor agnostic APIs. A new map and unmap ops were added to
rte_busstructure. Implementation of those was done currently only on the PCI bus. The implementation takes the driver map and umap implementation as bypassto the VFIO mapping.
Quoted 2 lines
That is, in case of no specific map/unmap from the PCI driver, VFIO mapping, if possible, will be used.
This paragraph should be rewritten to better fit a commit log.
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Application use with those APIs is quite simple:
* allocate memory
* take a device, and query itsrte_device.
* call the bus map function for this device.Is the device already configured with the existing mappings? Should the application stop it before attempting to map its allocated memory?
Am not following.
When the application wants to register new memory for DMA for this device it calls map. When it wants to unregister it calls unmap. w/o explicit call to the map function the memory cannot be used for DMA.
Quoted 39 lines
Quoted 3 lines
Future work will deprecate the
rte_vfio_dma_mapand
rte_vfio_dma_unmapQuoted 5 lines
APIs, leaving the PCI device APIs as the preferred option for the user.
Signed-off-by Shahaf Shuler <[email protected]>--- drivers/bus/pci/pci_common.c | 78++++++++++++++++++++++++++++
Quoted 29 lines
drivers/bus/pci/rte_bus_pci.h | 14 +++++ lib/librte_eal/common/eal_common_bus.c | 22 ++++++++ lib/librte_eal/common/include/rte_bus.h | 57 ++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 2 + 5 files changed, 173 insertions(+)(quoted diff) +0
-0
b/drivers/bus/pci/pci_common.cindex6276e5d695..018080c48b100644drivers/bus/pci/pci_common.c +15
-0@@ -528,6 +528,82 @@ pci_unplug(struct rte_device *dev)return ret; } +/** +* DMA Map memory segment to device. After a successful call the +device +* will be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be mapped. +* @param iova +* Starting IOVA address of memory to be mapped. +* @param len +* Length of memory segment being mapped. +* @return +* - 0 On success.
Quoted 8 lines
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 547
Quoted 2 lines
drivers/bus/pci/pci_common.c +2
-0+* - Negative value and rte_errno is set otherwise. +*/ This doc should be on the callback typedef, not their implementation. The
rte_errnoerror spec should also be documented higher-up in the abstraction pile, on the bus callback I think. Everyone should follow the same error codes for applications to really be able to use any implementation generically.
OK.
Quoted 8 lines
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 548
Quoted 2 lines
drivers/bus/pci/pci_common.c +1
-0+static int __rte_experimentalThe
__rte_experimentalis not necessary in compilation units themselves, only in the headers.In any case, it would only be the publicly available API that must be marked as such, so more the callback typedefs than their implementations.
OK
Quoted 8 lines
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 552
Quoted 6 lines
drivers/bus/pci/pci_common.c +5
-0+pci_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t +len) { + struct rte_pci_device *pdev = RTE_DEV_TO_PCI(dev); + +if (!pdev || !pdev->driver) {pdev cannot be null here, nor should its driver be.
So you say to relay on it and drop the check?
Quoted 110 lines
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev)
Quoted 12 lines
drivers/bus/pci/pci_common.c +11
-0+ rte_errno =EINVAL ; +return -rte_errno; + } +if (pdev->driver->map) +return pdev->driver->map (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) + return rte_vfio_container_dma_map(-1, (uintptr_t)addr,iova,
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 565
Quoted 1 line
drivers/bus/pci/pci_common.c +1
-0+ len);Reiterating:
RTE_VFIO_DEFAULT_CONTAINER_FDis more readable I think than
-1 here.Quoted 12 lines
drivers/bus/pci/pci_common.c +12
-0+ rte_errno =ENOTSUP ; +return -rte_errno; +} + +/** +* Un-map memory segment to device. After a successful call the +device +* will not be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 585
Quoted 10 lines
drivers/bus/pci/pci_common.c +10
-0+* Starting virtual address of memory to be unmapped. +* @param iova +* Starting IOVA address of memory to be unmapped. +* @param len +* Length of memory segment being unmapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/ +static int __rte_experimentalSame as before for
__rte_experimentaland doc.Quoted 6 lines
drivers/bus/pci/pci_common.c +6
-0+pci_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, +size_t len) { + struct rte_pci_device *pdev = RTE_DEV_TO_PCI(dev); + +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ;drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev)
Quoted 10 lines
drivers/bus/pci/pci_common.c +10
-0+return -rte_errno; + } +if (pdev->driver->unmap) +return pdev->driver->unmap (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) + return rte_vfio_container_dma_unmap(-1, (uintptr_t)addr,iova,
Quoted 42 lines
drivers/bus/pci/pci_common.c +7
-0+ len); + rte_errno =ENOTSUP ; +return -rte_errno; +} +struct rte_pci_bus rte_pci_bus = { .bus = { .scan = rte_pci_scan,@@ -536,6 +612,8 @@ struct rte_pci_bus rte_pci_bus = { .plug = pci_plug, .unplug = pci_unplug, .parse = pci_parse, + .map = pci_dma_map, + .unmap = pci_dma_unmap, .get_iommu_class = rte_pci_get_iommu_class, .dev_iterate = rte_pci_dev_iterate, .hot_unplug_handler = pci_hot_unplug_handler, diff --git
a/drivers/bus/pci/rte_bus_pci.hb/drivers/bus/pci/rte_bus_pci.hindexf0d6d81c00..00b2d412c7100644drivers/bus/pci/rte_bus_pci.h +12
-0@@ -114,6 +114,18 @@ typedef int (pci_probe_t)(struct rte_pci_driver *, struct rte_pci_device *); typedef int (pci_remove_t)(structrte_pci_device *);/** +* Driver-specific DMA mapping. +*/ +typedef int (pci_dma_map_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** +* Driver-specific DMA unmapping. +*/ +typedef int (pci_dma_unmap_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** * A structure describing a PCI driver. */ struct
rte_pci_driver{drivers/bus/pci/rte_bus_pci.h struct rte_pci_driver { · line 138
Quoted 5 lines
drivers/bus/pci/rte_bus_pci.h +2
-0@@ -122,6 +134,8 @@ struct rte_pci_driver {struct rte_pci_bus *bus;/**< PCI bus reference. */ pci_probe_t *probe;/**< Device Probe function. */ pci_remove_t *remove;/**< Device Remove function. */ +pci_dma_map_t *map;/**< device dma map function. */ +pci_dma_unmap_t *unmap;/**< device dma unmap function. */
Quoted 2 lines
I'd call both callbacks
dma_mapanddma_unmap. It's clearer and more consistent.
OK.
Quoted 28 lines
Quoted 28 lines
drivers/bus/pci/rte_bus_pci.h +0
-0const struct rte_pci_id *id_table;/**< ID table, NULL terminated. */ uint32_t drv_flags;/**< Flags RTE_PCI_DRV_*. */ };lib/librte_eal/common/eal_common_bus.c +15
-0@@ -285,3 +285,25 @@ rte_bus_sigbus_handler(const void *failure_addr)return ret; } + +int __rte_experimental +rte_bus_dma_map (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->map ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -rte_errno; + } +return dev->bus->map (dev, addr, iova, len); } + +int __rte_experimental +rte_bus_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{
Quoted 8 lines
lib/librte_eal/common/eal_common_bus.c rte_bus_sigbus_handler(const void *failure_addr)
Quoted 5 lines
lib/librte_eal/common/eal_common_bus.c +5
-0+if (dev->bus->unmap ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -rte_errno; + } +return dev->bus->unmap (dev, addr, iova, len); }These functions should be called rte_dev_dma_{map,unmap} and be part of
eal_common_dev.cinstead.
Will move.
Quoted 59 lines
Quoted 48 lines
lib/librte_eal/common/include/rte_bus.h +38
-0@@ -168,6 +168,48 @@ typedef int (*rte_bus_unplug_t)(struct rte_device *dev); typedef int (*rte_bus_parse_t)(const char *name, void *addr);/** +* Bus specific DMA map function. +* After a successful call, the memory segment will be mapped to the +* given device. +* +* @param dev +* Device pointer. +* @param addr +* Virtual address to map. +* @param iova +* IOVA address to map. +* @param len +* Length of the memory segment being mapped. +* +* @return +* 0 if mapping was successful. +* Negative value and rte_errno is set otherwise. +*/ +typedef int (*rte_bus_map_t )(struct rte_device *dev,void *addr, +uint64_t iova,size_t len); + +/** +* Bus specific DMA unmap function. +* After a successful call, the memory segment will no longer be +* accessible by the given device. +* +* @param dev +* Device pointer. +* @param addr +* Virtual address to unmap. +* @param iova +* IOVA address to unmap. +* @param len +* Length of the memory segment being mapped. +* +* @return +* 0 if un-mapping was successful. +* Negative value and rte_errno is set otherwise. +*/ lib/librte_eal/common/include/rte_bus.h typedef int (*rte_bus_unplug_t)(struct rte_device
Quoted 10 lines
lib/librte_eal/common/include/rte_bus.h +4
-1+typedef int (*rte_bus_unmap_t )(struct rte_device *dev,void *addr, +uint64_t iova,size_t len); + +/** * Implement a specific hot-unplug handler, which is responsible for * handle the failure when device be hot-unplugged. When the event of * hot-unplug be detected, it could call this function to handle @@ -238,6 +280,8 @@ struct rte_bus { rte_bus_plug_t plug; /**< Probe single device for drivers */ rte_bus_unplug_t unplug;/**< Remove single device from driver */
Quoted 7 lines
lib/librte_eal/common/include/rte_bus.h struct rte_bus {
Quoted 3 lines
lib/librte_eal/common/include/rte_bus.h +2
-0rte_bus_parse_t parse;/**< Parse a device name */ +rte_bus_map_t map;/**< DMA map for device in the bus */ +rte_bus_unmap_t unmap;/**< DMA unmap for device in the bus */
Same as for the driver callbacks,
dma_mapanddma_unmapseem a better fit for the field names.
OK
Quoted 50 lines
lib/librte_eal/common/include/rte_bus.h
Quoted 3 lines
lib/librte_eal/common/include/rte_bus.h +0
-0struct rte_bus_conf conf;/**< Bus configuration */ rte_bus_get_iommu_class_t get_iommu_class;/**< Get iommu class */
Quoted 2 lines
rte_dev_iterate_tdev_iterate; /**< Device iterator. */ @@ -356,6
+400,19 @@ structrte_bus*rte_bus_find_by_name(const char*busname);
Quoted 7 lines
enum
rte_iova_moderte_bus_get_iommu_class(void);/**
+ * Wrapper to call the bus specific DMA map function.
+ */lib/librte_eal/common/include/rte_bus.h +2
-0+int __rte_experimental +rte_bus_dma_map(struct rte_device *dev, void *addr, uint64_t iova,lib/librte_eal/common/include/rte_bus.h struct rte_bus *rte_bus_find_by_name(const char *busname); · line 415
Quoted 10 lines
lib/librte_eal/common/include/rte_bus.h +10
-0+size_t len); + +/** +* Wrapper to call the bus specific DMA unmap function. +*/ +int __rte_experimental +rte_bus_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova, +size_t len); + +/** Same as earlier -> these seem device-level functions, not bus-related. You won't map those addresses to all devices on the bus.
Quoted 16 lines
* Helper for Bus registration.
* The constructor has higher priority than PMD constructors. */lib/librte_eal/rte_eal_version.map +2
-1@@ -364,4 +364,6 @@ EXPERIMENTAL { rte_service_may_be_active; rte_socket_count; rte_socket_id_by_idx; + rte_bus_dma_map; + rte_bus_dma_unmap; }; --Quoted 2 lines
2.12.0
--
Gaëtan Rivet
6WIND
Re: [PATCH 3/6] bus: introduce DMA memory mapping for external memory
Quoted 32 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:17 PM, Gaëtan Rivet:
Quoted 31 lines
Subject: Re: [PATCH 3/6] bus: introduce DMA memory mapping for external memory
Quoted 1 line · Shahaf Shuler
The DPDK APIs expose 3 different modes to work with memory used for
DMA:
Quoted 2 lines
1. Use the DPDK owned memory (backed by the DPDK provided
hugepages).
Quoted 2 lines
This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the
DPDK
Quoted 18 lines
layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
Quoted 9 lines
Quoted 2 lines
How are those other vendors' devices mapped initially right now? Are they using #2 scheme instead? Then the user will remap everything using #3?
It is not a re-map, it is a completely different mode for the memory management.
The first question to ask is "how the application wants to manage its memory" ?
If it is either #1 or #2 above, no problem to make the mapping internal on the "other vendor devices" as they can register to the memory event callback which trigger every time new memory is added to the DPDK memory management system. For #3 the memory does not exists in the DPDK memory management system, and no memory events. Hence the application needs to explicitly call the dma MAP.
The change on this patch is just to make it more generic than calling only VFIO.
Right! I mostly used #1 ports and never really thought about other kind of memory management or how they might follow a different logic.
Do you think this could be used with a lot of sequential mapping/unmappings happening?
I'm thinking for example about a crypto app feeding crypto buffers, being able to directly map the result instead of copying it within buffers might be interesting. But then you'd have to unmap often.
- Is the unmap() simple from the app PoV?
- Must the mapping remain available for a long time?
- Does the app need to call tx_descriptor_status() a few times or does dma_unmap() verify that the mapping is not in use before unmapping?
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH 3/6] bus: introduce DMA memory mapping for external memory
Thursday, February 14, 2019 4:01 PM, Gaëtan Rivet:
Quoted 7 lines
Subject: Re: [PATCH 3/6] bus: introduce DMA memory mapping for external memory
Quoted 3 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:17 PM, Gaëtan Rivet:
Quoted 2 lines
Subject: Re: [PATCH 3/6] bus: introduce DMA memory mapping for external memory
[...]
Quoted 17 lines
Quoted 3 lines
Quoted 3 lines
How are those other vendors' devices mapped initially right now? Are they using #2 scheme instead? Then the user will remap everything using
#3?
Quoted 2 lines
It is not a re-map, it is a completely different mode for the memory
management.
Quoted 1 line
The first question to ask is "how the application wants to manage its
memory" ?
Quoted 1 line
If it is either #1 or #2 above, no problem to make the mapping internal on
the "other vendor devices" as they can register to the memory event callback which trigger every time new memory is added to the DPDK memory management system.
Quoted 1 line
For #3 the memory does not exists in the DPDK memory management
system, and no memory events. Hence the application needs to explicitly call the dma MAP.
Quoted 1 line
The change on this patch is just to make it more generic than calling only
Quoted 8 lines
VFIO.
Quoted 1 line
Right! I mostly used #1 ports and never really thought about other kind of memory management or how they might follow a different logic.
Do you think this could be used with a lot of sequential mapping/unmappings happening?
It much depends how efficient is the driver mapping and unmapping. In most cases, mapping is heavy operation.
Quoted 6 lines
I'm thinking for example about a crypto app feeding crypto buffers, being able to directly map the result instead of copying it within buffers might be interesting. But then you'd have to unmap often.
- Is the
unmap()simple from the app PoV?
Yes, just call rte_bus_dma_unmap.
Quoted 2 lines
- Must the mapping remain available for a long time?
It must remain as long as you need the device to access the memory. On your example, it should remain till the crypto dev finished writing the buffers.
Quoted 4 lines
- Does the app need to call
tx_descriptor_status()a few times or doesdma_unmap()verify that the mapping is not in use before
unmapping?
I think it is a matter of driver implementation.
In general, it is application responsibly to make sure the memory is no longer needed before unmapping, just like I don't destroy today mempool being used by some rxq. It can be done by any means the application has, not only tx_descriptor_status. Driver can protect bad application and warn + fail the call on such case, however it is not a must.
Quoted 4 lines
--
Gaëtan Rivet
6WIND
PATCH 4/6 net/mlx5: refactor external memory registration
In reply to Shahaf ShulerMove the memory region creation to a separate function to prepare the ground for the reuse of it on the PCI driver map and unmap functions.
--- drivers/net/mlx5/mlx5_mr.c | 86 +++++++++++++++++++++++++++-------------- 1 file changed, 57 insertions(+), 29 deletions(-)
drivers/net/mlx5/mlx5_mr.c +57 -29
@@ -1109,6 +1109,58 @@ mlx5_mr_flush_local_cache(struct mlx5_mr_ctrl *mr_ctrl) }/** +* Creates a memory region for external memory, that is memory which is not +* part of the DPDK memory segments. +* +* @param dev +* Pointer to the ethernet device. +* @param addr +* Starting virtual address of memory. +* @param len +* Length of memory segment being mapped. +* @param socked_id +* Socket to allocate heap memory for the control structures. +* +* @return +* Pointer to MR structure on success, NULL otherwise. +*/ +static struct mlx5_mr * +mlx5_create_mr_ext (struct rte_eth_dev *dev,uintptr_t addr,size_t len, +int socket_id) +{ +struct priv *priv = dev->data->dev_private; +struct mlx5_mr *mr =NULL ; + + mr =rte_zmalloc_socket (NULL , +RTE_ALIGN_CEIL (sizeof (*mr), +RTE_CACHE_LINE_SIZE ), +RTE_CACHE_LINE_SIZE , socket_id); +if (mr ==NULL ) +return NULL ; + mr->ibv_mr = mlx5_glue->reg_mr (priv->pd, (void *)addr, len, +IBV_ACCESS_LOCAL_WRITE ); +if (mr->ibv_mr ==NULL ) { +DRV_LOG (WARNING , +"port %u fail to create a verbs MR for address (%p)" , + dev->data->port_id, (void *)addr); +rte_free (mr); +return NULL ; + } + mr->msl =NULL ;/* Mark it is external memory. */ + mr->ms_bmp =NULL ; + mr->ms_n =1 ; + mr->ms_bmp_n =1 ; +DRV_LOG (DEBUG , +"port %u MR CREATED (%p) for external memory %p:\n" +" [0x%" PRIxPTR", 0x%" PRIxPTR")," +" lkey=0x%x base_idx=%u ms_n=%u, ms_bmp_n=%u" , + dev->data->port_id, (void *)mr, (void *)addr, + addr, addr + len,rte_cpu_to_be_32 (mr->ibv_mr->lkey), + mr->ms_base_idx, mr->ms_n, mr->ms_bmp_n); +return mr; +} + +/** * Called during rte_mempool_mem_iter() by mlx5_mr_update_ext_mp(). * * Externally allocated chunk is registered and a MR is created for the chunk.
@@ -1142,43 +1194,19 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque,rte_rwlock_read_unlock (&priv->mr.rwlock);if (lkey !=UINT32_MAX )return ; - mr =rte_zmalloc_socket (NULL , -RTE_ALIGN_CEIL (sizeof (*mr), -RTE_CACHE_LINE_SIZE ), -RTE_CACHE_LINE_SIZE , mp->socket_id); -if (mr ==NULL ) { -DRV_LOG (WARNING , -"port %u unable to allocate memory for a new MR of" -" mempool (%s)." , - dev->data->port_id, mp->name); - data->ret =-1 ; -return ; - }DRV_LOG (DEBUG ,"port %u register MR for chunk #%d of mempool (%s)" , dev->data->port_id, mem_idx, mp->name); - mr->ibv_mr = mlx5_glue->reg_mr (priv->pd, (void *)addr, len, -IBV_ACCESS_LOCAL_WRITE ); -if (mr->ibv_mr ==NULL ) { + mr =mlx5_create_mr_ext (dev, addr, len, mp->socket_id); +if (!mr) {DRV_LOG (WARNING , -"port %u fail to create a verbs MR for address (%p)" , - dev->data->port_id, (void *)addr); -rte_free (mr); +"port %u unable to allocate a new MR of" +" mempool (%s)." , + dev->data->port_id, mp->name); data->ret =-1 ;return ; } - mr->msl =NULL ;/* Mark it is external memory. */ - mr->ms_bmp =NULL ; - mr->ms_n =1 ; - mr->ms_bmp_n =1 ;rte_rwlock_write_lock (&priv->mr.rwlock);LIST_INSERT_HEAD (&priv->mr.mr_list, mr, mr); -DRV_LOG (DEBUG , -"port %u MR CREATED (%p) for external memory %p:\n" -" [0x%" PRIxPTR", 0x%" PRIxPTR")," -" lkey=0x%x base_idx=%u ms_n=%u, ms_bmp_n=%u" , - dev->data->port_id, (void *)mr, (void *)addr, - addr, addr + len,rte_cpu_to_be_32 (mr->ibv_mr->lkey), - mr->ms_base_idx, mr->ms_n, mr->ms_bmp_n);/* Insert to the global cache table. */ mr_insert_dev_cache (dev, mr);rte_rwlock_write_unlock (&priv->mr.rwlock);
Signature · 1 line
2.12.0
PATCH 5/6 net/mlx5: support PCI device DMA map and unmap
In reply to Shahaf ShulerThe implementation reuses the external memory registration work done by commit[1].
Note about representors:
The current representor design will not work
with those map and unmap functions. The reason is that for representors we have multiple IB devices share the same PCI function, so mapping will happen only on one of the representors and not all of them.
While it is possible to implement such support, the IB representor design is going to be changed during DPDK19.05. The new design will have a single IB device for all representors, hence sharing of a single memory region between all representors will be possible.
[1]
commit 7e43a32ee060
("net/mlx5: support externally allocated static memory")
--- drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 146 ++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 5 ++ 3 files changed, 153 insertions(+)
drivers/net/mlx5/mlx5.c +2 -0
@@ -1626,6 +1626,8 @@ static struct rte_pci_driver mlx5_driver = { .id_table = mlx5_pci_id_map, .probe = mlx5_pci_probe, .remove = mlx5_pci_remove, + .map = mlx5_dma_map, + .unmap = mlx5_dma_unmap, .drv_flags = (RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV | RTE_PCI_DRV_PROBE_AGAIN), };
drivers/net/mlx5/mlx5_mr.c +146 -0
158 lines not on this page · raw message
drivers/net/mlx5/mlx5_rxtx.h +5 -0
@@ -28,6 +28,7 @@#include <rte_atomic.h> #include <rte_spinlock.h> #include <rte_io.h> +#include <rte_bus_pci.h> #include "mlx5_utils.h" #include "mlx5.h"
@@ -367,6 +368,10 @@ uint32_t mlx5_rx_addr2mr_bh(struct mlx5_rxq_data *rxq, uintptr_t addr);uint32_t mlx5_tx_mb2mr_bh (struct mlx5_txq_data *txq,struct rte_mbuf *mb);uint32_t mlx5_tx_update_ext_mp (struct mlx5_txq_data *txq,uintptr_t addr,struct rte_mempool *mp); +int mlx5_dma_map (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len); +int mlx5_dma_unmap (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len);/** * Provide safe 64bit store operation to mlx5 UAR region for both 32bit and
Signature · 1 line
2.12.0
Re: [PATCH 5/6] net/mlx5: support PCI device DMA map and unmap
Quoted 82 lines · Shahaf Shuler
The implementation reuses the external memory registration work done by commit[1].
Note about representors:
The current representor design will not work
with those map and unmap functions. The reason is that for representors we have multiple IB devices share the same PCI function, so mapping will happen only on one of the representors and not all of them.While it is possible to implement such support, the IB representor design is going to be changed during DPDK19.05. The new design will have a single IB device for all representors, hence sharing of a single memory region between all representors will be possible.
[1]
commit7e43a32ee060
("net/mlx5: support externally allocated static memory")Signed-off-by Shahaf Shuler <[email protected]>--- drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 146 ++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 5 ++ 3 files changed, 153 insertions(+)drivers/net/mlx5/mlx5.c +2
-0@@ -1626,6 +1626,8 @@ static struct rte_pci_driver mlx5_driver = { .id_table = mlx5_pci_id_map, .probe = mlx5_pci_probe, .remove = mlx5_pci_remove, + .map = mlx5_dma_map, + .unmap = mlx5_dma_unmap, .drv_flags = (RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV | RTE_PCI_DRV_PROBE_AGAIN), };drivers/net/mlx5/mlx5_mr.c +28
-0@@ -14,6 +14,7 @@#include <rte_mempool.h> #include <rte_malloc.h> #include <rte_rwlock.h> +#include <rte_bus_pci.h> #include "mlx5.h" #include "mlx5_mr.h" @@ -1215,6 +1216,151 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, }/** +* Finds the first ethdev that match the pci device. +* The existence of multiple ethdev per pci device is only with representors. +* On such case, it is enough to get only one of the ports as they all share +* the same ibv context. +* +* @param pdev +* Pointer to the PCI device. +* +* @return +* Pointer to the ethdev if found, NULL otherwise. +*/ +static struct rte_eth_dev * +pci_dev_to_eth_dev (struct rte_pci_device *pdev) +{ +struct rte_eth_dev *dev; +const char *drv_name; +uint16_t port_id =0 ; + +/** +* We really need to iterate all eth devices regardless of +* their owner. +*/ +while (port_id <RTE_MAX_ETHPORTS ) { + port_id =rte_eth_find_next (port_id); +if (port_id >=RTE_MAX_ETHPORTS ) +break ; + dev = &rte_eth_devices[port_id];
drivers/net/mlx5/mlx5_mr.c mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, · line 1255
Quoted 10 lines
drivers/net/mlx5/mlx5_mr.c +10
-0+ drv_name = dev->device->driver->name; +if (!strncmp (drv_name,MLX5_DRIVER_NAME , +sizeof (MLX5_DRIVER_NAME ) +1 ) && + pdev ==RTE_DEV_TO_PCI (dev->device)) { +/* found the PCI device. */ +return dev; + } + } +return NULL ; +}
Might I interest you in the new API?
{
struct rte_dev_iterator it;
struct rte_device *dev; RTE_DEV_FOREACH(dev, "class=eth", &it)
if (dev == &pdev->device)
return it.class_device;
return NULL;}
Quoted 137 lines
drivers/net/mlx5/mlx5_mr.c +108
-0+ +/** +* DPDK callback to DMA map external memory to a PCI device. +* +* @param pdev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be mapped. +* @param iova +* Starting IOVA address of memory to be mapped. +* @param len +* Length of memory segment being mapped. +* +* @return +* 0 on success, negative value on error. +*/ +int +mlx5_dma_map (struct rte_pci_device *pdev,void *addr, + uint64_t iova __rte_unused, size_t len) +{ +struct rte_eth_dev *dev; +struct mlx5_mr *mr; +struct priv *priv; + + dev =pci_dev_to_eth_dev (pdev); +if (!dev) { +DRV_LOG (WARNING ,"unable to find matching ethdev " +"to PCI device %p" , (void *)pdev); +return -1 ; + } + priv = dev->data->dev_private; + mr =mlx5_create_mr_ext (dev, (uintptr_t )addr, len,SOCKET_ID_ANY ); +if (!mr) { +DRV_LOG (WARNING , +"port %u unable to dma map" , dev->data->port_id); +return -1 ; + } +rte_rwlock_write_lock (&priv->mr.rwlock); +LIST_INSERT_HEAD (&priv->mr.mr_list, mr, mr); +/* Insert to the global cache table. */ +mr_insert_dev_cache (dev, mr); +rte_rwlock_write_unlock (&priv->mr.rwlock); +return 0 ; +} + +/** +* DPDK callback to DMA unmap external memory to a PCI device. +* +* @param pdev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be unmapped. +* @param iova +* Starting IOVA address of memory to be unmapped. +* @param len +* Length of memory segment being unmapped. +* +* @return +* 0 on success, negative value on error. +*/ +int +mlx5_dma_unmap (struct rte_pci_device *pdev,void *addr, + uint64_t iova __rte_unused, size_t len __rte_unused) +{ +struct rte_eth_dev *dev; +struct priv *priv; +struct mlx5_mr *mr; +struct mlx5_mr_cache entry; + + dev =pci_dev_to_eth_dev (pdev); +if (!dev) { +DRV_LOG (WARNING ,"unable to find matching ethdev " +"to PCI device %p" , (void *)pdev); +return -1 ; + } + priv = dev->data->dev_private; +rte_rwlock_read_lock (&priv->mr.rwlock); + mr =mr_lookup_dev_list (dev, &entry, (uintptr_t )addr); +if (!mr) { +DRV_LOG (WARNING ,"address 0x%" PRIxPTR" wasn't registered " +"to PCI device %p" , (uintptr_t )addr, + (void *)pdev); +rte_rwlock_read_unlock (&priv->mr.rwlock); +return -1 ; + } +LIST_REMOVE (mr, mr); +LIST_INSERT_HEAD (&priv->mr.mr_free_list, mr, mr); +DEBUG ("port %u remove MR(%p) from list" , dev->data->port_id, + (void *)mr); +mr_rebuild_dev_cache (dev); +/* +* Flush local caches by propagating invalidation across cores. +* rte_smp_wmb() is enough to synchronize this event. If one of +* freed memsegs is seen by other core, that means the memseg +* has been allocated by allocator, which will come after this +* free call. Therefore, this store instruction (incrementing +* generation below) will be guaranteed to be seen by other core +* before the core sees the newly allocated memory. +*/ + ++priv->mr.dev_gen; +DEBUG ("broadcasting local cache flush, gen=%d" , + priv->mr.dev_gen); +rte_smp_wmb (); +rte_rwlock_read_unlock (&priv->mr.rwlock); +return 0 ; +} + +/** * Register MR for entire memory chunks in a Mempool having externally allocated * memory and fill in local cache. * drivers/net/mlx5/mlx5_rxtx.h +5
-0@@ -28,6 +28,7 @@#include <rte_atomic.h> #include <rte_spinlock.h> #include <rte_io.h> +#include <rte_bus_pci.h> #include "mlx5_utils.h" #include "mlx5.h" @@ -367,6 +368,10 @@ uint32_t mlx5_rx_addr2mr_bh(struct mlx5_rxq_data *rxq, uintptr_t addr);uint32_t mlx5_tx_mb2mr_bh (struct mlx5_txq_data *txq,struct rte_mbuf *mb);uint32_t mlx5_tx_update_ext_mp (struct mlx5_txq_data *txq,uintptr_t addr,struct rte_mempool *mp); +int mlx5_dma_map (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len); +int mlx5_dma_unmap (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len);/** * Provide safe 64bit store operation to mlx5 UAR region for both 32bit and Signature · 1 line
2.12.0
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH 5/6] net/mlx5: support PCI device DMA map and unmap
Quoted 100 lines · Gaëtan Rivet
Quoted 82 lines · Shahaf Shuler
The implementation reuses the external memory registration work done by commit[1].
Note about representors:
The current representor design will not work
with those map and unmap functions. The reason is that for representors we have multiple IB devices share the same PCI function, so mapping will happen only on one of the representors and not all of them.While it is possible to implement such support, the IB representor design is going to be changed during DPDK19.05. The new design will have a single IB device for all representors, hence sharing of a single memory region between all representors will be possible.
[1]
commit7e43a32ee060
("net/mlx5: support externally allocated static memory")Signed-off-by Shahaf Shuler <[email protected]>--- drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 146 ++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 5 ++ 3 files changed, 153 insertions(+)drivers/net/mlx5/mlx5.c +2
-0@@ -1626,6 +1626,8 @@ static struct rte_pci_driver mlx5_driver = { .id_table = mlx5_pci_id_map, .probe = mlx5_pci_probe, .remove = mlx5_pci_remove, + .map = mlx5_dma_map, + .unmap = mlx5_dma_unmap, .drv_flags = (RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV | RTE_PCI_DRV_PROBE_AGAIN), };drivers/net/mlx5/mlx5_mr.c +28
-0@@ -14,6 +14,7 @@#include <rte_mempool.h> #include <rte_malloc.h> #include <rte_rwlock.h> +#include <rte_bus_pci.h> #include "mlx5.h" #include "mlx5_mr.h" @@ -1215,6 +1216,151 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, }/** +* Finds the first ethdev that match the pci device. +* The existence of multiple ethdev per pci device is only with representors. +* On such case, it is enough to get only one of the ports as they all share +* the same ibv context. +* +* @param pdev +* Pointer to the PCI device. +* +* @return +* Pointer to the ethdev if found, NULL otherwise. +*/ +static struct rte_eth_dev * +pci_dev_to_eth_dev (struct rte_pci_device *pdev) +{ +struct rte_eth_dev *dev; +const char *drv_name; +uint16_t port_id =0 ; + +/** +* We really need to iterate all eth devices regardless of +* their owner. +*/ +while (port_id <RTE_MAX_ETHPORTS ) { + port_id =rte_eth_find_next (port_id); +if (port_id >=RTE_MAX_ETHPORTS ) +break ; + dev = &rte_eth_devices[port_id];drivers/net/mlx5/mlx5_mr.c mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, · line 1255
Quoted 10 lines
drivers/net/mlx5/mlx5_mr.c +10
-0+ drv_name = dev->device->driver->name; +if (!strncmp (drv_name,MLX5_DRIVER_NAME , +sizeof (MLX5_DRIVER_NAME ) +1 ) && + pdev ==RTE_DEV_TO_PCI (dev->device)) { +/* found the PCI device. */ +return dev; + } + } +return NULL ; +}Might I interest you in the new API?
{ struct rte_dev_iterator it; struct rte_device *dev;
Quoted 6 lines
RTE_DEV_FOREACH(dev, "class=eth", &it) if (dev == &pdev->device) return it.class_device; return NULL;}
On that note, this could be in the PCI bus instead?
Quoted 141 lines
Quoted 137 lines
drivers/net/mlx5/mlx5_mr.c +108
-0+ +/** +* DPDK callback to DMA map external memory to a PCI device. +* +* @param pdev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be mapped. +* @param iova +* Starting IOVA address of memory to be mapped. +* @param len +* Length of memory segment being mapped. +* +* @return +* 0 on success, negative value on error. +*/ +int +mlx5_dma_map (struct rte_pci_device *pdev,void *addr, + uint64_t iova __rte_unused, size_t len) +{ +struct rte_eth_dev *dev; +struct mlx5_mr *mr; +struct priv *priv; + + dev =pci_dev_to_eth_dev (pdev); +if (!dev) { +DRV_LOG (WARNING ,"unable to find matching ethdev " +"to PCI device %p" , (void *)pdev); +return -1 ; + } + priv = dev->data->dev_private; + mr =mlx5_create_mr_ext (dev, (uintptr_t )addr, len,SOCKET_ID_ANY ); +if (!mr) { +DRV_LOG (WARNING , +"port %u unable to dma map" , dev->data->port_id); +return -1 ; + } +rte_rwlock_write_lock (&priv->mr.rwlock); +LIST_INSERT_HEAD (&priv->mr.mr_list, mr, mr); +/* Insert to the global cache table. */ +mr_insert_dev_cache (dev, mr); +rte_rwlock_write_unlock (&priv->mr.rwlock); +return 0 ; +} + +/** +* DPDK callback to DMA unmap external memory to a PCI device. +* +* @param pdev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be unmapped. +* @param iova +* Starting IOVA address of memory to be unmapped. +* @param len +* Length of memory segment being unmapped. +* +* @return +* 0 on success, negative value on error. +*/ +int +mlx5_dma_unmap (struct rte_pci_device *pdev,void *addr, + uint64_t iova __rte_unused, size_t len __rte_unused) +{ +struct rte_eth_dev *dev; +struct priv *priv; +struct mlx5_mr *mr; +struct mlx5_mr_cache entry; + + dev =pci_dev_to_eth_dev (pdev); +if (!dev) { +DRV_LOG (WARNING ,"unable to find matching ethdev " +"to PCI device %p" , (void *)pdev); +return -1 ; + } + priv = dev->data->dev_private; +rte_rwlock_read_lock (&priv->mr.rwlock); + mr =mr_lookup_dev_list (dev, &entry, (uintptr_t )addr); +if (!mr) { +DRV_LOG (WARNING ,"address 0x%" PRIxPTR" wasn't registered " +"to PCI device %p" , (uintptr_t )addr, + (void *)pdev); +rte_rwlock_read_unlock (&priv->mr.rwlock); +return -1 ; + } +LIST_REMOVE (mr, mr); +LIST_INSERT_HEAD (&priv->mr.mr_free_list, mr, mr); +DEBUG ("port %u remove MR(%p) from list" , dev->data->port_id, + (void *)mr); +mr_rebuild_dev_cache (dev); +/* +* Flush local caches by propagating invalidation across cores. +* rte_smp_wmb() is enough to synchronize this event. If one of +* freed memsegs is seen by other core, that means the memseg +* has been allocated by allocator, which will come after this +* free call. Therefore, this store instruction (incrementing +* generation below) will be guaranteed to be seen by other core +* before the core sees the newly allocated memory. +*/ + ++priv->mr.dev_gen; +DEBUG ("broadcasting local cache flush, gen=%d" , + priv->mr.dev_gen); +rte_smp_wmb (); +rte_rwlock_read_unlock (&priv->mr.rwlock); +return 0 ; +} + +/** * Register MR for entire memory chunks in a Mempool having externally allocated * memory and fill in local cache. * drivers/net/mlx5/mlx5_rxtx.h +5
-0@@ -28,6 +28,7 @@#include <rte_atomic.h> #include <rte_spinlock.h> #include <rte_io.h> +#include <rte_bus_pci.h> #include "mlx5_utils.h" #include "mlx5.h" @@ -367,6 +368,10 @@ uint32_t mlx5_rx_addr2mr_bh(struct mlx5_rxq_data *rxq, uintptr_t addr);uint32_t mlx5_tx_mb2mr_bh (struct mlx5_txq_data *txq,struct rte_mbuf *mb);uint32_t mlx5_tx_update_ext_mp (struct mlx5_txq_data *txq,uintptr_t addr,struct rte_mempool *mp); +int mlx5_dma_map (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len); +int mlx5_dma_unmap (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len);/** * Provide safe 64bit store operation to mlx5 UAR region for both 32bit and Signature · 1 line
2.12.0Signature · 2 lines
Gaëtan Rivet 6WIND
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH 5/6] net/mlx5: support PCI device DMA map and unmap
Wednesday, February 13, 2019 1:44 PM, Gaëtan Rivet:
Quoted 5 lines
Subject: Re: [PATCH 5/6] net/mlx5: support PCI device DMA map and unmap
Quoted 2 lines · Gaëtan Rivet
Quoted 1 line · Shahaf Shuler
The implementation reuses the external memory registration work done
[..]
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
drivers/net/mlx5/mlx5_mr.c +14
-0+ +/** +* We really need to iterate all eth devices regardless of +* their owner. +*/ +while (port_id <RTE_MAX_ETHPORTS ) { + port_id =rte_eth_find_next (port_id); +if (port_id >=RTE_MAX_ETHPORTS ) +break ; + dev = &rte_eth_devices[port_id]; + drv_name = dev->device->driver->name; +if (!strncmp (drv_name,MLX5_DRIVER_NAME , +sizeof (MLX5_DRIVER_NAME ) +1 ) && + pdev ==RTE_DEV_TO_PCI (dev->device)) {
Quoted 8 lines
Quoted 8 lines
drivers/net/mlx5/mlx5_mr.c mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, · line 1255
Quoted 6 lines
drivers/net/mlx5/mlx5_mr.c +6
-0+/* found the PCI device. */ +return dev; + } + } +return NULL ; +}Might I interest you in the new API?
Good suggestion, will have a look on it in depth.
Quoted 5 lines
Quoted 5 lines
{ struct rte_dev_iterator it; struct rte_device *dev;
Quoted 8 lines
Quoted 6 lines
RTE_DEV_FOREACH(dev, "class=eth", &it) if (dev == &pdev->device) return it.class_device; return NULL;}
On that note, this could be in the PCI bus instead?
We can put it on the PCI bus, but it would mean the PCI bus will not be device agnostic.
Currently, I couldn't find any reference to eth_dev on the PCI bus, besides a single macro which convert to pci device that doesn't really do type checks.
Having it in, would mean the PCI will need start to distinguish between ethdev, crypto dev and what ever devices exists on its bus.
Quoted 152 lines
Quoted 24 lines
Quoted 24 lines
drivers/net/mlx5/mlx5_mr.c +23
-0+ +/** +* DPDK callback to DMA map external memory to a PCI device. +* +* @param pdev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be mapped. +* @param iova +* Starting IOVA address of memory to be mapped. +* @param len +* Length of memory segment being mapped. +* +* @return +* 0 on success, negative value on error. +*/ +int +mlx5_dma_map (struct rte_pci_device *pdev,void *addr, + uint64_t iova __rte_unused, size_t len) { +struct rte_eth_dev *dev; +struct mlx5_mr *mr; +struct priv *priv; +Quoted 8 lines
drivers/net/mlx5/mlx5_mr.c mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque,
Quoted 8 lines
drivers/net/mlx5/mlx5_mr.c +8
-0+ dev =pci_dev_to_eth_dev (pdev); +if (!dev) { +DRV_LOG (WARNING ,"unable to find matching ethdev " +"to PCI device %p" , (void *)pdev); +return -1 ; + } + priv = dev->data->dev_private; + mr = mlx5_create_mr_ext(dev, (uintptr_t)addr, len,
SOCKET_ID_ANY);Quoted 39 lines
Quoted 39 lines
drivers/net/mlx5/mlx5_mr.c +39
-0+if (!mr) { +DRV_LOG (WARNING , +"port %u unable to dma map" , dev->data->port_id); +return -1 ; + } +rte_rwlock_write_lock (&priv->mr.rwlock); +LIST_INSERT_HEAD (&priv->mr.mr_list, mr, mr); +/* Insert to the global cache table. */ +mr_insert_dev_cache (dev, mr); +rte_rwlock_write_unlock (&priv->mr.rwlock); +return 0 ; +} + +/** +* DPDK callback to DMA unmap external memory to a PCI device. +* +* @param pdev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be unmapped. +* @param iova +* Starting IOVA address of memory to be unmapped. +* @param len +* Length of memory segment being unmapped. +* +* @return +* 0 on success, negative value on error. +*/ +int +mlx5_dma_unmap (struct rte_pci_device *pdev,void *addr, + uint64_t iova __rte_unused, size_t len __rte_unused) { +struct rte_eth_dev *dev; +struct priv *priv; +struct mlx5_mr *mr; +struct mlx5_mr_cache entry; + + dev =pci_dev_to_eth_dev (pdev); +if (!dev) { + DRV_LOG(WARNING, "unable to find matching ethdev "Quoted 8 lines
drivers/net/mlx5/mlx5_mr.c mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque,
Quoted 8 lines
drivers/net/mlx5/mlx5_mr.c +8
-0+ "to PCI device %p", (void *)pdev); +return -1 ; + } + priv = dev->data->dev_private; +rte_rwlock_read_lock (&priv->mr.rwlock); + mr =mr_lookup_dev_list (dev, &entry, (uintptr_t )addr); + if (!mr) { + DRV_LOG(WARNING, "address 0x%" PRIxPTR " wasn'tregistered "
Quoted 21 lines
Quoted 21 lines
drivers/net/mlx5/mlx5_mr.c +21
-0+"to PCI device %p" , (uintptr_t )addr, + (void *)pdev); +rte_rwlock_read_unlock (&priv->mr.rwlock); +return -1 ; + } +LIST_REMOVE (mr, mr); +LIST_INSERT_HEAD (&priv->mr.mr_free_list, mr, mr); +DEBUG ("port %u remove MR(%p) from list" , dev->data->port_id, + (void *)mr); +mr_rebuild_dev_cache (dev); +/* +* Flush local caches by propagating invalidation across cores. +* rte_smp_wmb() is enough to synchronize this event. If one of +* freed memsegs is seen by other core, that means the memseg +* has been allocated by allocator, which will come after this +* free call. Therefore, this store instruction (incrementing +* generation below) will be guaranteed to be seen by other core +* before the core sees the newly allocated memory. +*/ + ++priv->mr.dev_gen; + DEBUG("broadcasting local cache flush, gen=%d",Quoted 8 lines
drivers/net/mlx5/mlx5_mr.c mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque,
Quoted 8 lines
drivers/net/mlx5/mlx5_mr.c +7
-0+ priv->mr.dev_gen); +rte_smp_wmb (); +rte_rwlock_read_unlock (&priv->mr.rwlock); +return 0 ; +} + +/** * Register MR for entire memory chunks in a Mempool having externally allocated
Quoted 12 lines
Quoted 12 lines
* memory and fill in local cache.
*(quoted diff) +0
-0
b/drivers/net/mlx5/mlx5_rxtx.hindexc2529f96bc..f3f84dbac3100644drivers/net/mlx5/mlx5_rxtx.h +1
-0@@ -28,6 +28,7 @@#include <rte_atomic.h> #include <rte_spinlock.h> #include <rte_io.h> +#include <rte_bus_pci.h> Quoted 5 lines
Quoted 5 lines
drivers/net/mlx5/mlx5_rxtx.h +0
-0#include "mlx5_utils.h" #include "mlx5.h" @@ -367,6 +368,10 @@ uint32_t mlx5_rx_addr2mr_bh(struct
mlx5_rxq_data*rxq,uintptr_taddr);uint32_tmlx5_tx_mb2mr_bh(struct mlx5_txq_data *txq, struct rte_mbuf *mb);
uint32_tmlx5_tx_update_ext_mp(structmlx5_txq_data*txq,uintptr_taddr,Quoted 2 lines
drivers/net/mlx5/mlx5_rxtx.h
Quoted 2 lines
drivers/net/mlx5/mlx5_rxtx.h +1
-0struct rte_mempool *mp); +int mlx5_dma_map(struct rte_pci_device *pdev, void *addr, uint64_tiova,
Quoted 2 lines
drivers/net/mlx5/mlx5_rxtx.h
Quoted 2 lines
drivers/net/mlx5/mlx5_rxtx.h +2
-0+ size_t len); +int mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, uint64_tiova,
Quoted 12 lines
Quoted 8 lines
drivers/net/mlx5/mlx5_rxtx.h +1
-0+ size_t len);/** * Provide safe 64bit store operation to mlx5 UAR region for both 32bit and
--
2.12.0--
Gaëtan Rivet
6WIND--
Gaëtan Rivet
6WIND
Re: [PATCH 5/6] net/mlx5: support PCI device DMA map and unmap
Quoted 46 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:44 PM, Gaëtan Rivet:
Quoted 5 lines
Subject: Re: [PATCH 5/6] net/mlx5: support PCI device DMA map and unmap
Quoted 2 lines · Gaëtan Rivet
Quoted 1 line · Shahaf Shuler
The implementation reuses the external memory registration work done
[..]
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
drivers/net/mlx5/mlx5_mr.c +14
-0+ +/** +* We really need to iterate all eth devices regardless of +* their owner. +*/ +while (port_id <RTE_MAX_ETHPORTS ) { + port_id =rte_eth_find_next (port_id); +if (port_id >=RTE_MAX_ETHPORTS ) +break ; + dev = &rte_eth_devices[port_id]; + drv_name = dev->device->driver->name; +if (!strncmp (drv_name,MLX5_DRIVER_NAME , +sizeof (MLX5_DRIVER_NAME ) +1 ) && + pdev ==RTE_DEV_TO_PCI (dev->device)) {Quoted 8 lines
Quoted 8 lines
drivers/net/mlx5/mlx5_mr.c mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, · line 1255
Quoted 6 lines
drivers/net/mlx5/mlx5_mr.c +6
-0+/* found the PCI device. */ +return dev; + } + } +return NULL ; +}Might I interest you in the new API?
Good suggestion, will have a look on it in depth.
Quoted 12 lines
Quoted 11 lines
{ struct rte_dev_iterator it; struct rte_device *dev;RTE_DEV_FOREACH(dev, "class=eth", &it) if (dev == &pdev->device) return it.class_device; return NULL;}
Quoted 7 lines
Quoted 1 line
On that note, this could be in the PCI bus instead?
We can put it on the PCI bus, but it would mean the PCI bus will not be device agnostic.
Currently, I couldn't find any reference toeth_devon the PCI bus, besides a single macro which convert to pci device that doesn't really do type checks.Having it in, would mean the PCI will need start to distinguish between ethdev, crypto dev and what ever devices exists on its bus.
I think it's worth thinking about it.
It can stay class-agnostic:
void *
rte_pci_device_class(struct rte_pci_device *pdev, const char *class)
{
char devstr[15+strlen(class)];
struct rte_dev_iterator it;
struct rte_device *dev;
snprintf(devstr, sizeof(devstr), "bus=pci/class=%s", class);
RTE_DEV_FOREACH(dev, devstr, &it)
if (dev == &pdev->device)
return it.class_device;
return NULL;
}(not a fan of the stack VLA but whatever.)
then:
eth_dev = rte_pci_device_class(pdev, "eth");
Whichever type of device could be returned. Only limit is that you have to know beforehand what is the device type of the PCI device you are querying about, but that's necessary anyway.
And if it was instead a crypto dev, it would return NULL.
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH 5/6] net/mlx5: support PCI device DMA map and unmap
Thursday, February 14, 2019 12:22 PM, Gaëtan Rivet:
Quoted 52 lines
Subject: Re: [PATCH 5/6] net/mlx5: support PCI device DMA map and unmap
Quoted 49 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:44 PM, Gaëtan Rivet:
Quoted 7 lines
Subject: Re: [PATCH 5/6] net/mlx5: support PCI device DMA map and unmap
Quoted 3 lines · Gaëtan Rivet
Quoted 2 lines · Shahaf Shuler
The implementation reuses the external memory registration work done
[..]
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
drivers/net/mlx5/mlx5_mr.c +14
-0+ +/** +* We really need to iterate all eth devices regardless of +* their owner. +*/ +while (port_id <RTE_MAX_ETHPORTS ) { + port_id =rte_eth_find_next (port_id); +if (port_id >=RTE_MAX_ETHPORTS ) +break ; + dev = &rte_eth_devices[port_id]; + drv_name = dev->device->driver->name; +if (!strncmp (drv_name,MLX5_DRIVER_NAME , +sizeof (MLX5_DRIVER_NAME ) +1 ) && + pdev ==RTE_DEV_TO_PCI (dev->device)) {Quoted 8 lines
Quoted 8 lines
drivers/net/mlx5/mlx5_mr.c mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, · line 1255
Quoted 6 lines
drivers/net/mlx5/mlx5_mr.c +6
-0+/* found the PCI device. */ +return dev; + } + } +return NULL ; +}Might I interest you in the new API?
Good suggestion, will have a look on it in depth.
Quoted 13 lines
Quoted 11 lines
{ struct rte_dev_iterator it; struct rte_device *dev;RTE_DEV_FOREACH(dev, "class=eth", &it) if (dev == &pdev->device) return it.class_device; return NULL;}
On that note, this could be in the PCI bus instead?
Looking in more depth into it. it looks like ethdev is the only device class which register its type. So putting a generic iterator for every class on the PCI bus looks an overkill to me at this point.
I think I will take the above suggestion to replace the internal PMD code.
Quoted 41 lines
Quoted 2 lines
We can put it on the PCI bus, but it would mean the PCI bus will not be
device agnostic.
Quoted 1 line
Currently, I couldn't find any reference to
eth_devon the PCI bus, besidesa single macro which convert to pci device that doesn't really do type checks.
Quoted 2 lines
Having it in, would mean the PCI will need start to distinguish between
ethdev, crypto dev and what ever devices exists on its bus.
Quoted 1 line
I think it's worth thinking about it.
It can stay class-agnostic:void * rte_pci_device_class(struct rte_pci_device *pdev, const char *class) { char devstr[15+strlen(class)]; struct rte_dev_iterator it; struct rte_device *dev; snprintf(devstr, sizeof(devstr), "bus=pci/class=%s", class); RTE_DEV_FOREACH(dev, devstr, &it) if (dev == &pdev->device) return it.class_device; return NULL; }(not a fan of the stack VLA but whatever.)
then:eth_dev = rte_pci_device_class(pdev, "eth");Whichever type of device could be returned. Only limit is that you have to know beforehand what is the device type of the PCI device you are querying about, but that's necessary anyway.
And if it was instead a crypto dev, it would return NULL.
--
Gaëtan Rivet
6WIND
PATCH 6/6 doc: deprecate VFIO DMA map APIs
In reply to Shahaf ShulerAs those have been replaced by rte_bus_dma_map and rte_pci_dma_unmap APIs.
--- doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 ++++ lib/librte_eal/common/include/rte_vfio.h | 6 ++++-- 3 files changed, 9 insertions(+), 3 deletions(-)
doc/guides/prog_guide/env_abstraction_layer.rst +1 -1
@@ -282,7 +282,7 @@ The expected workflow is as follows: - If IOVA table is not specified, IOVA addresses will be assumed to be unavailable - Other processes must attach to the memory area before they can use it -* Perform DMA mapping with ``rte_vfio_dma_map`` if needed +* Perform DMA mapping with ``rte_bus_dma_map`` if needed * Use the memory area in your application * If memory area is no longer needed, it can be unregistered - If the area was mapped for DMA, unmapping must be performed before
doc/guides/rel_notes/deprecation.rst +4 -0
@@ -35,6 +35,10 @@ Deprecation Notices + ``rte_eal_devargs_type_count`` +* vfio: removal of ``rte_vfio_dma_map`` and ``rte_vfio_dma_unmap`` APIs which + have been replaced with ``rte_bus_dma_map`` and ``rte_bus_dma_unmap`` + functions. The due date for the removal targets DPDK 19.08. + * pci: Several exposed functions are misnamed. The following functions are deprecated starting from v17.11 and are replaced:
lib/librte_eal/common/include/rte_vfio.h +4 -2
@@ -188,6 +188,7 @@ int rte_vfio_clear_group(int vfio_group_fd);/** +* @deprecated * Map memory region for use with VFIO. * * @note Require at least one device to be attached at the time of
@@ -208,11 +209,12 @@ rte_vfio_clear_group(int vfio_group_fd);* 0 if success. * -1 on error. */ -int +int __rte_deprecatedrte_vfio_dma_map (uint64_t vaddr,uint64_t iova,uint64_t len);/** +* @deprecated * Unmap memory region from VFIO. * * @param vaddr
@@ -229,7 +231,7 @@ rte_vfio_dma_map(uint64_t vaddr, uint64_t iova, uint64_t len);* -1 on error. */ -int +int __rte_deprecatedrte_vfio_dma_unmap (uint64_t vaddr,uint64_t iova,uint64_t len);/** * Parse IOMMU group number for a device
Signature · 1 line
2.12.0
Quoted 14 lines · Shahaf Shuler
This series is in continue to RFC[1].
The DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
Quoted 8 lines
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Why can not we have case 2 covering case 3?
Quoted 5 lines
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
As you say, VFIO is common, and when allowing DMAs programmed in user space, the right thing to do. I'm assuming there is an IOMMU hardware and this is what Mellanox and NXP rely on in some way or another.
Having each driver doing things in their own way will end up in a harder to validate system. If there is an IOMMU hardware, same mechanism should be used always, leaving to the IOMMU hw specific implementation to deal with the details. If a NIC is IOMMU-able, that should not be supported by specific vendor drivers but through a generic solution like VFIO which will validate a device with such capability and to perform the required actions for that case. VFIO and IOMMU should be modified as needed for supporting this requirement instead of leaving vendor drivers to implement their own solution.
In any case, I think this support should be in a different patchset than the private user space mappings.
Quoted 43 lines
The work in this patch moves the DMA mapping to vendor agnostic APIs. A new map and unmap ops were added to
rte_busstructure. Implementation of those was done currently only on the PCI bus. The implementation takes the driver map and umap implementation as bypass to the VFIO mapping. That is, in case of no specific map/unmap from the PCI driver, VFIO mapping, if possible, will be used.Application use with those APIs is quite simple:
* allocate memory
* take a device, and query itsrte_device.
* call the bus map function for this device.Future work will deprecate the
rte_vfio_dma_mapandrte_vfio_dma_unmapAPIs, leaving the PCI device APIs as the preferred option for the user.[1] https://patches.dpdk.org/patch/47796/
Shahaf Shuler (6):
vfio: allow DMA map of memory for the default vfio fd
vfio: don't fail to DMA map if memory is already mapped
bus: introduce DMA memory mapping for external memory net/mlx5: refactor external memory registration net/mlx5: support PCI device DMA map and unmap
doc: deprecate VFIO DMA map APIsdoc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 + drivers/bus/pci/pci_common.c | 78 +++++++ drivers/bus/pci/rte_bus_pci.h | 14 ++ drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 232 ++++++++++++++++--- drivers/net/mlx5/mlx5_rxtx.h | 5 + lib/librte_eal/common/eal_common_bus.c | 22 ++ lib/librte_eal/common/include/rte_bus.h | 57 +++++ lib/librte_eal/common/include/rte_vfio.h | 12 +- lib/librte_eal/linuxapp/eal/eal_vfio.c | 26 ++- lib/librte_eal/rte_eal_version.map | 2 + 12 files changed, 418 insertions(+), 38 deletions(-)--
2.12.0
Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
Quoted 28 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 3 lines · Shahaf Shuler
This series is in continue to RFC[1].
The DPDK APIs expose 3 different modes to work with memory used for
DMA:
Quoted 2 lines
1. Use the DPDK owned memory (backed by the DPDK provided
hugepages).
Quoted 2 lines
This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the
DPDK
Quoted 12 lines
layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
Quoted 4 lines
Quoted 3 lines
The scope of the patch focus on #3 above.
Why can not we have case 2 covering case 3?
Because it is not our choice rather the DPDK application.
We could not allow it, and force the application to register their external memory to the DPDK memory management system. However IMO it will be wrong.
The use case exists - some application wants to manage their memory by themselves. w/o the extra overhead of rte_malloc, without creating a special socket to populate the memory and without redundant API calls to rte_extmem_*.
Simply allocate chunk of memory, DMA map it to device and that’s it.
Quoted 9 lines
Quoted 7 lines
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).As you say, VFIO is common, and when allowing DMAs programmed in user space, the right thing to do.
It is common indeed. Why it the right thing to do?
I'm assuming there is an IOMMU hardware and
Quoted 1 line
this is what Mellanox and NXP rely on in some way or another.
For Mellanox, the device works with virtual memory, not physical. If you think of it, it is more secure for user space application. Mellanox device has internal memory translation unit between virtual memory and physical memory. IOMMU can be added on top of it, in case the host doesn't trust the device or the device is given to untrusted entity like VM.
Quoted 3 lines
Having each driver doing things in their own way will end up in a harder to validate system.
Different vendors will have different HW implementations. We cannot force everybody to align the IOMMU.
What we can do, is to ease the user life and provide vendor agnostic APIs which just provide the needed functionality. On our case DMA map and unmap. The user should not care if its IOMMU, Mellanox memory registration through verbs or NXP special mapping.
The sys admin should set/unset the IOMMU as a general mean of protection. And this of course will work also w/ Mellanox devices.
If there is an IOMMU hardware, same mechanism should be
Quoted 7 lines
used always, leaving to the IOMMU hw specific implementation to deal with the details. If a NIC is IOMMU-able, that should not be supported by specific vendor drivers but through a generic solution like VFIO which will validate a device with such capability and to perform the required actions for that case. VFIO and IOMMU should be modified as needed for supporting this requirement instead of leaving vendor drivers to implement their own solution.
Again - I am against of forcing every PCI device to use VFIO, and I don't think IOMMU as a HW device should control other PCI devices. I see nothing wrong with device which also has extra capabilities of memory translation, and adds another level of security to the user application.
Quoted 63 lines
In any case, I think this support should be in a different patchset than the private user space mappings.
Quoted 4 lines
The work in this patch moves the DMA mapping to vendor agnostic APIs. A new map and unmap ops were added to
rte_busstructure. Implementation of those was done currently only on the PCI bus. The implementation takes the driver map and umap implementation as bypassto the VFIO mapping.
Quoted 9 lines
That is, in case of no specific map/unmap from the PCI driver, VFIO mapping, if possible, will be used.
Application use with those APIs is quite simple:
* allocate memory
* take a device, and query itsrte_device.
* call the bus map function for this device.Future work will deprecate the
rte_vfio_dma_mapand
rte_vfio_dma_unmapQuoted 4 lines
APIs, leaving the PCI device APIs as the preferred option for the user.
[1]
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpat
Quoted 1 line
ches.dpdk.org%2Fpatch%2F47796%2F&data=02%7C01%7Cshahafs%40 mellanox
Quoted 1 line
.com%7Cdc209a2ceace48c0452008d691a8762d%7Ca652971c7d2e4d9ba6a4d1 49256f
Quoted 1 line
461b%7C0%7C0%7C636856550053348339&sdata=3TEUJfS9jBOsbvaPYwo itQLj7o
Quoted 28 lines
h9VCrtaK7We%2FItg5c%3D&reserved=0
Shahaf Shuler (6):
vfio: allow DMA map of memory for the default vfio fd
vfio: don't fail to DMA map if memory is already mapped
bus: introduce DMA memory mapping for external memory net/mlx5: refactor external memory registration net/mlx5: support PCI device DMA map and unmap
doc: deprecate VFIO DMA map APIsdoc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 + drivers/bus/pci/pci_common.c | 78 +++++++ drivers/bus/pci/rte_bus_pci.h | 14 ++ drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 232 ++++++++++++++++--- drivers/net/mlx5/mlx5_rxtx.h | 5 + lib/librte_eal/common/eal_common_bus.c | 22 ++ lib/librte_eal/common/include/rte_bus.h | 57 +++++ lib/librte_eal/common/include/rte_vfio.h | 12 +- lib/librte_eal/linuxapp/eal/eal_vfio.c | 26 ++- lib/librte_eal/rte_eal_version.map | 2 + 12 files changed, 418 insertions(+), 38 deletions(-)--
2.12.0
Quoted 32 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
Quoted 31 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 3 lines · Shahaf Shuler
This series is in continue to RFC[1].
The DPDK APIs expose 3 different modes to work with memory used for
DMA:
Quoted 2 lines
1. Use the DPDK owned memory (backed by the DPDK provided
hugepages).
Quoted 2 lines
This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the
DPDK
Quoted 15 lines
layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Quoted 7 lines
Quoted 1 line
Why can not we have case 2 covering case 3?
Because it is not our choice rather the DPDK application.
We could not allow it, and force the application to register their external memory to the DPDK memory management system. However IMO it will be wrong.
The use case exists - some application wants to manage their memory by themselves. w/o the extra overhead ofrte_malloc, without creating a special socket to populate the memory and without redundant API calls to rte_extmem_*.Simply allocate chunk of memory, DMA map it to device and that’s it.
Just a small note: while this sounds good on paper, i should point out that at least *registering* the memory with DPDK is a necessity. You may see rte_extmem_* calls as redundant (and i agree, to an extent), but we don't advertise our PMD's capabilities in a way that makes it easy to determine whether a particular PMD will or will not work without registering external memory within DPDK (i.e. does it use rte_virt2memseg() internally, for example).
So, extmem register calls are a necessary evil in such case, and IMO should be called out as required for such external memory usage scenario.
Signature · 2 lines
Thanks, Anatoly
Thursday, February 14, 2019 12:19 PM, Burakov, Anatoly:
Quoted 58 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 5 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
Quoted 4 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
On Wed, Feb 13, 2019 at 9:11 AM Shahaf Shuler
Quoted 24 lines
Quoted 24 lines
Quoted 3 lines · wrote:
This series is in continue to RFC[1].
The DPDK APIs expose 3 different modes to work with memory used for
DMA:
Quoted 2 lines
1. Use the DPDK owned memory (backed by the DPDK provided
hugepages).
Quoted 2 lines
This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the
DPDK
Quoted 12 lines
layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
The scope of the patch focus on #3 above.
Why can not we have case 2 covering case 3?
Because it is not our choice rather the DPDK application.
We could not allow it, and force the application to register their externalmemory to the DPDK memory management system. However IMO it will be wrong.
Quoted 1 line
The use case exists - some application wants to manage their memory by
themselves. w/o the extra overhead of
rte_malloc, without creating a special socket to populate the memory and without redundant API calls to rte_extmem_*.Quoted 2 lines
Simply allocate chunk of memory, DMA map it to device and that’s it.
Just a small note: while this sounds good on paper, i should point out that at least *registering* the memory with DPDK is a necessity. You may see rte_extmem_* calls as redundant (and i agree, to an extent), but we don't advertise our PMD's capabilities in a way that makes it easy to determine whether a particular PMD will or will not work without registering external memory within DPDK (i.e. does it use
rte_virt2memseg()internally, for example).
Quoted 2 lines
So, extmem register calls are a necessary evil in such case, and IMO should be called out as required for such external memory usage scenario.
If we are going to force all to use the extmem, then there is no need w/ this API. we can have the PMDs to register when the memory is registered. We can just drop the vfio_dma_map APIs and that's it.
Quoted 4 lines
--
Thanks,
Anatoly
Quoted 59 lines · Shahaf Shuler
Thursday, February 14, 2019 12:19 PM, Burakov, Anatoly:
Quoted 58 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 5 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
Quoted 4 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
On Wed, Feb 13, 2019 at 9:11 AM Shahaf Shuler
Quoted 24 lines
Quoted 24 lines
Quoted 3 lines · wrote:
This series is in continue to RFC[1].
The DPDK APIs expose 3 different modes to work with memory used for
DMA:
Quoted 2 lines
1. Use the DPDK owned memory (backed by the DPDK provided
hugepages).
Quoted 2 lines
This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the
DPDK
Quoted 12 lines
layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
The scope of the patch focus on #3 above.
Why can not we have case 2 covering case 3?
Because it is not our choice rather the DPDK application.
We could not allow it, and force the application to register their externalmemory to the DPDK memory management system. However IMO it will be wrong.
Quoted 1 line
The use case exists - some application wants to manage their memory by
themselves. w/o the extra overhead of
rte_malloc, without creating a special socket to populate the memory and without redundant API calls to rte_extmem_*.Quoted 2 lines
Simply allocate chunk of memory, DMA map it to device and that’s it.
Just a small note: while this sounds good on paper, i should point out that at least *registering* the memory with DPDK is a necessity. You may see rte_extmem_* calls as redundant (and i agree, to an extent), but we don't advertise our PMD's capabilities in a way that makes it easy to determine whether a particular PMD will or will not work without registering external memory within DPDK (i.e. does it use
rte_virt2memseg()internally, for example).
Quoted 6 lines
Quoted 2 lines
So, extmem register calls are a necessary evil in such case, and IMO should be called out as required for such external memory usage scenario.
If we are going to force all to use the extmem, then there is no need w/ this API. we can have the PMDs to register when the memory is registered. We can just drop the
vfio_dma_mapAPIs and that's it.
Well, whether we needed it or not is not really my call, but what i can say is that using extmem_register is _necessary_ if you're going to use the PMD's. You're right, we could just map memory for DMA at register time - that would save one API call to get the memory working. It makes it a bit weird semantically, but i think we can live with that :)
Signature · 2 lines
Thanks, Anatoly
Thursday, February 14, 2019 6:20 PM, Burakov, Anatoly:
Quoted 13 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 9 lines · Shahaf Shuler
Thursday, February 14, 2019 12:19 PM, Burakov, Anatoly:
Quoted 8 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 4 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
Quoted 3 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
[...]
Quoted 11 lines
Quoted 2 lines
If we are going to force all to use the extmem, then there is no need w/
this API. we can have the PMDs to register when the memory is registered.
Quoted 2 lines
We can just drop the
vfio_dma_mapAPIs and that's it.Well, whether we needed it or not is not really my call, but what i can say is that using
extmem_registeris _necessary_ if you're going to use the PMD's. You're right, we could just map memory for DMA at register time - that would save one API call to get the memory working. It makes it a bit weird semantically, but i think we can live with that :)
This was not my suggestion 😊. I don't think the register API should do the mapping as well.
My thoughts were on one of the two options:
1. have the series I propose here, and enable the user to work with memory managed outside of DPDK. Either force the user to call rte_extmem_register before the mapping or devices which needs memory to be also registered in the DPDK system can fail the mapping.
2. not providing such option to application, and forcing applications to populate a socket w/ external memory.
I vote for #1.
Quoted 4 lines
--
Thanks,
Anatoly
Quoted 29 lines · Shahaf Shuler
Thursday, February 14, 2019 6:20 PM, Burakov, Anatoly:
Quoted 13 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 9 lines · Shahaf Shuler
Thursday, February 14, 2019 12:19 PM, Burakov, Anatoly:
Quoted 8 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 4 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
Quoted 3 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
[...]
Quoted 11 lines
Quoted 2 lines
If we are going to force all to use the extmem, then there is no need w/
this API. we can have the PMDs to register when the memory is registered.
Quoted 2 lines
We can just drop the
vfio_dma_mapAPIs and that's it.Well, whether we needed it or not is not really my call, but what i can say is that using
extmem_registeris _necessary_ if you're going to use the PMD's. You're right, we could just map memory for DMA at register time - that would save one API call to get the memory working. It makes it a bit weird semantically, but i think we can live with that :)
Quoted 6 lines
This was not my suggestion 😊. I don't think the register API should do the mapping as well.
My thoughts were on one of the two options:
1. have the series I propose here, and enable the user to work with memory managed outside of DPDK. Either force the user to callrte_extmem_registerbefore the mapping or devices which needs memory to be also registered in the DPDK system can fail the mapping.
2. not providing such option to application, and forcing applications to populate a socket w/ external memory.I vote for #1.
I too think #1 is better - we want this to be a valid use case. Allowing such usage in the first place is already gracious enough - all we ask in return is one extra API call, to politely let DPDK know that this memory exists and is going to be used for DMA :)
Also, having the memory registered will also allow us to refuse mapping if it cannot be found in DPDK maps - if rte_virt2memseg returns NULL, that means extmem_register was not called. I.e. we can _enforce_ usage of extmem_register, which i believe is a good thing for usability.
Quoted 5 lines
Quoted 5 lines
--
Thanks,
Anatoly
Signature · 2 lines
Thanks, Anatoly
Quoted 43 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
Quoted 28 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 3 lines · Shahaf Shuler
This series is in continue to RFC[1].
The DPDK APIs expose 3 different modes to work with memory used for
DMA:
Quoted 2 lines
1. Use the DPDK owned memory (backed by the DPDK provided
hugepages).
Quoted 2 lines
This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the
DPDK
Quoted 12 lines
layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
Quoted 4 lines
Quoted 3 lines
The scope of the patch focus on #3 above.
Why can not we have case 2 covering case 3?
Because it is not our choice rather the DPDK application.
We could not allow it, and force the application to register their external memory to the DPDK memory management system. However IMO it will be wrong.
The use case exists - some application wants to manage their memory by themselves. w/o the extra overhead ofrte_malloc, without creating a special socket to populate the memory and without redundant API calls to rte_extmem_*.
Quoted 3 lines
Simply allocate chunk of memory, DMA map it to device and that’s it.
Usability is a strong point, but up to some extent. DPDK is all about performance, and adding options the user can choose from will add pressure and complexity for keeping the performance. Your proposal makes sense from an user point of view, but will it avoid to modify things in the DPDK core for supporting this case broadly in the future? Multiprocess will be hard to get, if not impossible, without adding more complexity, and although you likely do not expect that use case requiring multiprocess support, once we have DPDK apps using this model, sooner or later those companies with products based on such option will demand broadly support. I can foresee not just multiprocess support will require changes in the future.
This reminds me the case of obtaining real time: the more complexity the less determinism can be obtained. It is not impossible, simply it is far more complex. Pure real time operating systems can add new functionalities, but it is hard to do it properly without jeopardising the main goal. Generic purpose operating systems can try to improve determinism, but up to some extent and with important complexity costs. DPDK is the real time operating system in this comparison.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
Quoted 6 lines
Quoted 2 lines
As you say, VFIO is common, and when allowing DMAs programmed in user space, the right thing to do.
It is common indeed. Why it the right thing to do?
Compared with UIO, for sure. VFIO does have the right view of the system in terms of which devices can properly be isolated. Can you confirm a specific implementation by a vendor can ensure same behaviour? If so, do you have duplicated code then? if the answer is your are using VFIO data, why not to use VFIO as the interface and add the required connection between VFIO and drivers?
What about mapping validation? is the driver doing that part or relying on kernel code? or is it just assuming the mapping is safe?
Quoted 11 lines
I'm assuming there is an IOMMU hardware and
Quoted 1 line
this is what Mellanox and NXP rely on in some way or another.
For Mellanox, the device works with virtual memory, not physical. If you think of it, it is more secure for user space application. Mellanox device has internal memory translation unit between virtual memory and physical memory.
IOMMU can be added on top of it, in case the host doesn't trust the device or the device is given to untrusted entity like VM.
Any current NIC or device will work with virtual addresses if IOMMU is in place, not matter if the device is IOMMU-aware or not. Any vendor, with that capability in their devices, should follow generic paths and a common interface with the vendor drivers being the executors. The drivers know how to tell the device, but they should be told what to tell and not by the user but by the kernel.
I think reading your comment "in case the host doesn't trust the device" makes easier to understand what you try to obtain, and at the same time makes my concerns not a problem at all. This is a case for DPDK being used in certain scenarios where the full system is trusted, what I think is a completely rightful option. My only concern then is the complexity it could imply sooner or later, although I have to admit it is not a strong one :-)
Quoted 27 lines
Quoted 2 lines
Having each driver doing things in their own way will end up in a harder
to
Quoted 1 line
validate system.
Different vendors will have different HW implementations. We cannot force everybody to align the IOMMU.
What we can do, is to ease the user life and provide vendor agnostic APIs which just provide the needed functionality. On our case DMA map and unmap. The user should not care if its IOMMU, Mellanox memory registration through verbs or NXP special mapping.The sys admin should set/unset the IOMMU as a general mean of protection. And this of course will work also w/ Mellanox devices.
If there is an IOMMU hardware, same mechanism should be
Quoted 2 lines
used always, leaving to the IOMMU hw specific implementation to deal with the details. If a NIC is IOMMU-able, that should not be supported by
specific
Quoted 1 line
vendor drivers but through a generic solution like VFIO which will
validate a
Quoted 1 line
device with such capability and to perform the required actions for that
case.
Quoted 3 lines
VFIO and IOMMU should be modified as needed for supporting this requirement instead of leaving vendor drivers to implement their own solution.
Quoted 7 lines
Again - I am against of forcing every PCI device to use VFIO, and I don't think IOMMU as a HW device should control other PCI devices. I see nothing wrong with device which also has extra capabilities of memory translation, and adds another level of security to the user application.
In a system with untrusted components using the device, a generic way of properly configure the system with the right protections should be used instead of relying on specific vendor implementation.
Quoted 67 lines
Quoted 2 lines
In any case, I think this support should be in a different patchset than
the
Quoted 26 lines
private user space mappings.
Quoted 4 lines
The work in this patch moves the DMA mapping to vendor agnostic APIs. A new map and unmap ops were added to
rte_busstructure. Implementation of those was done currently only on the PCI bus. The implementation takes the driver map and umap implementation as bypassto the VFIO mapping.
Quoted 9 lines
That is, in case of no specific map/unmap from the PCI driver, VFIO mapping, if possible, will be used.
Application use with those APIs is quite simple:
* allocate memory
* take a device, and query itsrte_device.
* call the bus map function for this device.Future work will deprecate the
rte_vfio_dma_mapand
rte_vfio_dma_unmapQuoted 4 lines
APIs, leaving the PCI device APIs as the preferred option for the user.
[1]
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpat
Quoted 1 line
Quoted 24 lines
ches.dpdk.org%2Fpatch%2F47796%2F&data=02%7C01%7Cshahafs%40 mellanox
Quoted 1 line
.com%7Cdc209a2ceace48c0452008d691a8762d%7Ca652971c7d2e4d9ba6a4d1 49256f
Quoted 1 line
461b%7C0%7C0%7C636856550053348339&sdata=3TEUJfS9jBOsbvaPYwo itQLj7o
Quoted 16 lines
h9VCrtaK7We%2FItg5c%3D&reserved=0
Shahaf Shuler (6):
vfio: allow DMA map of memory for the default vfio fd
vfio: don't fail to DMA map if memory is already mapped
bus: introduce DMA memory mapping for external memory net/mlx5: refactor external memory registration net/mlx5: support PCI device DMA map and unmap
doc: deprecate VFIO DMA map APIsdoc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 + drivers/bus/pci/pci_common.c | 78 +++++++ drivers/bus/pci/rte_bus_pci.h | 14 ++ drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 232++++++++++++++++---
Quoted 12 lines
Quoted 12 lines
drivers/net/mlx5/mlx5_rxtx.h | 5 + lib/librte_eal/common/eal_common_bus.c | 22 ++ lib/librte_eal/common/include/rte_bus.h | 57 +++++ lib/librte_eal/common/include/rte_vfio.h | 12 +- lib/librte_eal/linuxapp/eal/eal_vfio.c | 26 ++- lib/librte_eal/rte_eal_version.map | 2 + 12 files changed, 418 insertions(+), 38 deletions(-)--
2.12.0
Quoted 113 lines · Alejandro Lucero
Quoted 43 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
Quoted 28 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 3 lines · Shahaf Shuler
This series is in continue to RFC[1].
The DPDK APIs expose 3 different modes to work with memory used for
DMA:
Quoted 2 lines
1. Use the DPDK owned memory (backed by the DPDK provided
hugepages).
Quoted 2 lines
This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the
DPDK
Quoted 12 lines
layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
Quoted 4 lines
Quoted 3 lines
The scope of the patch focus on #3 above.
Why can not we have case 2 covering case 3?
Because it is not our choice rather the DPDK application.
We could not allow it, and force the application to register their external memory to the DPDK memory management system. However IMO it will be wrong.
The use case exists - some application wants to manage their memory by themselves. w/o the extra overhead ofrte_malloc, without creating a special socket to populate the memory and without redundant API calls to rte_extmem_*.Quoted 3 lines
Simply allocate chunk of memory, DMA map it to device and that’s it.
Usability is a strong point, but up to some extent. DPDK is all about performance, and adding options the user can choose from will add pressure and complexity for keeping the performance. Your proposal makes sense from an user point of view, but will it avoid to modify things in the DPDK core for supporting this case broadly in the future? Multiprocess will be hard to get, if not impossible, without adding more complexity, and although you likely do not expect that use case requiring multiprocess support, once we have DPDK apps using this model, sooner or later those companies with products based on such option will demand broadly support. I can foresee not just multiprocess support will require changes in the future.
This reminds me the case of obtaining real time: the more complexity the less determinism can be obtained. It is not impossible, simply it is far more complex. Pure real time operating systems can add new functionalities, but it is hard to do it properly without jeopardising the main goal. Generic purpose operating systems can try to improve determinism, but up to some extent and with important complexity costs. DPDK is the real time operating system in this comparison.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).Quoted 6 lines
Quoted 2 lines
As you say, VFIO is common, and when allowing DMAs programmed in user space, the right thing to do.
It is common indeed. Why it the right thing to do?
Compared with UIO, for sure. VFIO does have the right view of the system in terms of which devices can properly be isolated. Can you confirm a specific implementation by a vendor can ensure same behaviour? If so, do you have duplicated code then? if the answer is your are using VFIO data, why not to use VFIO as the interface and add the required connection between VFIO and drivers?
What about mapping validation? is the driver doing that part or relying on kernel code? or is it just assuming the mapping is safe?
Quoted 11 lines
I'm assuming there is an IOMMU hardware and
Quoted 1 line
this is what Mellanox and NXP rely on in some way or another.
For Mellanox, the device works with virtual memory, not physical. If you think of it, it is more secure for user space application. Mellanox device has internal memory translation unit between virtual memory and physical memory.
IOMMU can be added on top of it, in case the host doesn't trust the device or the device is given to untrusted entity like VM.Any current NIC or device will work with virtual addresses if IOMMU is in place, not matter if the device is IOMMU-aware or not. Any vendor, with that capability in their devices, should follow generic paths and a common interface with the vendor drivers being the executors. The drivers know how to tell the device, but they should be told what to tell and not by the user but by the kernel.
Quoted 8 lines
I think reading your comment "in case the host doesn't trust the device" makes easier to understand what you try to obtain, and at the same time makes my concerns not a problem at all. This is a case for DPDK being used in certain scenarios where the full system is trusted, what I think is a completely rightful option. My only concern then is the complexity it could imply sooner or later, although I have to admit it is not a strong one :-)
I forgot to mention the problem of leaving that option open in not fully trusted systems. I do not know how it could be avoided, maybe some checks in EAL initialization, but maybe this is not possible at all. Anyway, I think this is worth to be discussed further.
Quoted 109 lines
Quoted 28 lines
Quoted 2 lines
Having each driver doing things in their own way will end up in a
harder to
Quoted 1 line
validate system.
Different vendors will have different HW implementations. We cannot force everybody to align the IOMMU.
What we can do, is to ease the user life and provide vendor agnostic APIs which just provide the needed functionality. On our case DMA map and unmap. The user should not care if its IOMMU, Mellanox memory registration through verbs or NXP special mapping.The sys admin should set/unset the IOMMU as a general mean of protection. And this of course will work also w/ Mellanox devices.
If there is an IOMMU hardware, same mechanism should be
Quoted 1 line
used always, leaving to the IOMMU hw specific implementation to deal
with
Quoted 1 line
the details. If a NIC is IOMMU-able, that should not be supported by
specific
Quoted 1 line
vendor drivers but through a generic solution like VFIO which will
validate a
Quoted 1 line
device with such capability and to perform the required actions for
that case.
Quoted 3 lines
VFIO and IOMMU should be modified as needed for supporting this requirement instead of leaving vendor drivers to implement their own solution.
Quoted 7 lines
Again - I am against of forcing every PCI device to use VFIO, and I don't think IOMMU as a HW device should control other PCI devices. I see nothing wrong with device which also has extra capabilities of memory translation, and adds another level of security to the user application.
In a system with untrusted components using the device, a generic way of properly configure the system with the right protections should be used instead of relying on specific vendor implementation.
Quoted 68 lines
Quoted 2 lines
In any case, I think this support should be in a different patchset
than the
Quoted 13 lines
private user space mappings.
Quoted 4 lines
The work in this patch moves the DMA mapping to vendor agnostic APIs. A new map and unmap ops were added to
rte_busstructure. Implementation of those was done currently only on the PCI bus. The implementation takes the driver map and umap implementation as bypassto the VFIO mapping.
Quoted 3 lines
That is, in case of no specific map/unmap from the PCI driver, VFIO mapping, if possible, will be used.
Quoted 8 lines
Quoted 6 lines
Application use with those APIs is quite simple:
* allocate memory
* take a device, and query itsrte_device.
* call the bus map function for this device.Future work will deprecate the
rte_vfio_dma_mapand
rte_vfio_dma_unmapQuoted 1 line
APIs, leaving the PCI device APIs as the preferred option for the
user.
Quoted 5 lines
Quoted 3 lines
[1]
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpat
Quoted 1 line
Quoted 24 lines
ches.dpdk.org%2Fpatch%2F47796%2F&data=02%7C01%7Cshahafs%40 mellanox
Quoted 1 line
.com%7Cdc209a2ceace48c0452008d691a8762d%7Ca652971c7d2e4d9ba6a4d1 49256f
Quoted 1 line
461b%7C0%7C0%7C636856550053348339&sdata=3TEUJfS9jBOsbvaPYwo itQLj7o
Quoted 16 lines
h9VCrtaK7We%2FItg5c%3D&reserved=0
Shahaf Shuler (6):
vfio: allow DMA map of memory for the default vfio fd
vfio: don't fail to DMA map if memory is already mapped
bus: introduce DMA memory mapping for external memory net/mlx5: refactor external memory registration net/mlx5: support PCI device DMA map and unmap
doc: deprecate VFIO DMA map APIsdoc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 + drivers/bus/pci/pci_common.c | 78 +++++++ drivers/bus/pci/rte_bus_pci.h | 14 ++ drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 232++++++++++++++++---
Quoted 12 lines
Quoted 12 lines
drivers/net/mlx5/mlx5_rxtx.h | 5 + lib/librte_eal/common/eal_common_bus.c | 22 ++ lib/librte_eal/common/include/rte_bus.h | 57 +++++ lib/librte_eal/common/include/rte_vfio.h | 12 +- lib/librte_eal/linuxapp/eal/eal_vfio.c | 26 ++- lib/librte_eal/rte_eal_version.map | 2 + 12 files changed, 418 insertions(+), 38 deletions(-)--
2.12.0
Thursday, February 14, 2019 2:22 PM, Alejandro Lucero:
Quoted 39 lines · Shahaf Shuler
Wednesday, February 13, 2019 1:43 PM, Alejandro Lucero:
Quoted 28 lines
Subject: Re: [dpdk-dev] [PATCH 0/6] introduce DMA memory mapping for external memory
Quoted 3 lines · Shahaf Shuler
This series is in continue to RFC[1].
The DPDK APIs expose 3 different modes to work with memory used for
DMA:
Quoted 2 lines
1. Use the DPDK owned memory (backed by the DPDK provided
hugepages).
Quoted 2 lines
This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the
DPDK
Quoted 12 lines
layers.
2. Use memory allocated by the user and register to the DPDK memory systems. This is also referred as external memory. Upon registration of the external memory, the DPDK layers will DMA map it to all needed devices.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory. The user will need to explicitly call DMA map function in order to register such memory to the different devices.
Quoted 4 lines
Quoted 3 lines
The scope of the patch focus on #3 above.
Why can not we have case 2 covering case 3?
Because it is not our choice rather the DPDK application.
We could not allow it, and force the application to register their external memory to the DPDK memory management system. However IMO it will be wrong.
The use case exists - some application wants to manage their memory by themselves. w/o the extra overhead ofrte_malloc, without creating a special socket to populate the memory and without redundant API calls to rte_extmem_*.
Quoted 5 lines
Simply allocate chunk of memory, DMA map it to device and that’s it.
Usability is a strong point, but up to some extent. DPDK is all about performance, and adding options the user can choose from will add pressure and complexity for keeping the performance. Your proposal makes sense from an user point of view, but will it avoid to modify things in the DPDK core for supporting this case broadly in the future? Multiprocess will be hard to get, if not impossible, without adding more complexity, and although you likely do not expect that use case requiring multiprocess support, once we have DPDK apps using this model, sooner or later those companies with products based on such option will demand broadly support. I can foresee not just multiprocess support will require changes in the future.
This reminds me the case of obtaining real time: the more complexity the less determinism can be obtained. It is not impossible, simply it is far more complex. Pure real time operating systems can add new functionalities, but it is hard to do it properly without jeopardising the main goal. Generic purpose operating systems can try to improve determinism, but up to some extent and with important complexity costs. DPDK is the real time operating system in this comparison.
it makes some sense.
as I wrote to Anatoly, I am not against forcing the user to work only w/ DPDK registered memory.
we may cause some overhead to application, but will make things less complex. we just need to agree on it, and remove backdoors like vfio_dma_map (which BTW, currently being used by application, checkout VPP).
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
Quoted 8 lines
Quoted 2 lines
As you say, VFIO is common, and when allowing DMAs programmed in user space, the right thing to do.
It is common indeed. Why it the right thing to do?
Compared with UIO, for sure. VFIO does have the right view of the system in terms of which devices can properly be isolated. Can you confirm a specific implementation by a vendor can ensure same behaviour? If so, do you have duplicated code then? if the answer is your are using VFIO data, why not to use VFIO as the interface and add the required connection between VFIO and drivers?
What about mapping validation? is the driver doing that part or relying on kernel code? or is it just assuming the mapping is safe?
mapping validation is done by Mellanox kernel module, the kernel is trusted.
Quoted 8 lines
I'm assuming there is an IOMMU hardware and
Quoted 1 line
this is what Mellanox and NXP rely on in some way or another.
For Mellanox, the device works with virtual memory, not physical. If you think of it, it is more secure for user space application. Mellanox device has internal memory translation unit between virtual memory and physical memory. IOMMU can be added on top of it, in case the host doesn't trust the device or the device is given to untrusted entity like VM.
Any current NIC or device will work with virtual addresses if IOMMU is in place, not matter if the device is IOMMU-aware or not.
Not sure what you mean here. For example Intel devices works w/ VFIO and use iova to provide buffers to NIC. hence protection between multiple process is by application responsibility or new VFIO container. for devices which works w/ virtual addresses, sharing of device with multiple process is simple and secure.
Any vendor, with that capability in their devices, should follow generic paths and a common interface with the vendor drivers being the executors. The drivers know how to tell the device, but they should be told what to tell and not by the user but by the kernel.
Quoted 42 lines
I think reading your comment "in case the host doesn't trust the device" makes easier to understand what you try to obtain, and at the same time makes my concerns not a problem at all. This is a case for DPDK being used in certain scenarios where the full system is trusted, what I think is a completely rightful option. My only concern then is the complexity it could imply sooner or later, although I have to admit it is not a strong one :-)
Quoted 3 lines
Having each driver doing things in their own way will end up in a harder to validate system.
Different vendors will have different HW implementations. We cannot force everybody to align the IOMMU.
What we can do, is to ease the user life and provide vendor agnostic APIs which just provide the needed functionality. On our case DMA map and unmap. The user should not care if its IOMMU, Mellanox memory registration through verbs or NXP special mapping.The sys admin should set/unset the IOMMU as a general mean of protection. And this of course will work also w/ Mellanox devices.
If there is an IOMMU hardware, same mechanism should be
Quoted 7 lines
used always, leaving to the IOMMU hw specific implementation to deal with the details. If a NIC is IOMMU-able, that should not be supported by specific vendor drivers but through a generic solution like VFIO which will validate a device with such capability and to perform the required actions for that case. VFIO and IOMMU should be modified as needed for supporting this requirement instead of leaving vendor drivers to implement their own solution.
Again - I am against of forcing every PCI device to use VFIO, and I don't think IOMMU as a HW device should control other PCI devices. I see nothing wrong with device which also has extra capabilities of memory translation, and adds another level of security to the user application.
In a system with untrusted components using the device, a generic way of properly configure the system with the right protections should be used instead of relying on specific vendor implementation.
Quoted 14 lines
In any case, I think this support should be in a different patchset than the private user space mappings.
Quoted 4 lines
The work in this patch moves the DMA mapping to vendor agnostic APIs. A new map and unmap ops were added to
rte_busstructure. Implementation of those was done currently only on the PCI bus. The implementation takes the driver map and umap implementation as bypassto the VFIO mapping.
Quoted 3 lines
That is, in case of no specific map/unmap from the PCI driver, VFIO mapping, if possible, will be used.
Quoted 7 lines · Shahaf Shuler
Thursday, February 14, 2019 2:22 PM, Alejandro Lucero:
Quoted 1 line
Any current NIC or device will work with virtual addresses if IOMMU is
in place, not matter if the device is IOMMU-aware or not.
Not sure what you mean here. For example Intel devices works w/ VFIO and use iova to provide buffers to NIC. hence protection between multiple process is by application responsibility or new VFIO container.
As far as VFIO is concerned, "multiprocess protection" is not a thing, because the device cannot be used twice in the first place - each usage is strictly limited to one VFIO container. We just sidestep this "limitation" by sharing container/device file descriptors with multiple processes via IPC.
So while it's technically true that multiprocess protection is "application responsibility" because we can pass around fd's, it's still protected by the kernel. IOVA mappings are per-container, so the same IOVA range can be mapped twice (thrice...), as long as it's for a different set of devices, in effect making them virtual addresses.
Signature · 2 lines
Thanks, Anatoly
v2 cover introduce DMA memory mapping for external memory
In reply to Shahaf ShulerThe DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the rte_malloc header).
The user should create a memory, register it through rte_extmem_register API, and call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* call rte_extmem_register on the memory chunk.
* take a device, and query its rte_device.
* call the device specific mapping function for this device.
Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap APIs, leaving the rte device APIs as the preferred option for the user.
On v2:
- Added warn in release notes about the API change in vfio.
- Moved function doc to prototype declaration.
- Used dma_map and dma_unmap instead of map and unmap.
- Used RTE_VFIO_DEFAULT_CONTAINER_FD instead of -1 fixed value.
- Moved bus function to eal_common_dev.c and renamed them properly.
- Changed eth device iterator to use RTE_DEV_FOREACH.
- Enforced memory is registered with rte_extmem_* prior to mapping.
- Used EEXIST as the only possible return value from type1 vfio IOMMU mapping.
[1] https://patches.dpdk.org/patch/47796/
Shahaf Shuler (6):
vfio: allow DMA map of memory for the default vfio fd
vfio: don't fail to DMA map if memory is already mapped
bus: introduce device level DMA memory mapping net/mlx5: refactor external memory registration net/mlx5: support PCI device DMA map and unmap
doc: deprecate VFIO DMA map APIs
doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 + doc/guides/rel_notes/release_19_05.rst | 3 + drivers/bus/pci/pci_common.c | 48 ++++ drivers/bus/pci/rte_bus_pci.h | 40 ++++ drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 221 ++++++++++++++++--- drivers/net/mlx5/mlx5_rxtx.h | 5 + lib/librte_eal/common/eal_common_dev.c | 29 +++ lib/librte_eal/common/include/rte_bus.h | 44 ++++ lib/librte_eal/common/include/rte_dev.h | 43 ++++ lib/librte_eal/common/include/rte_vfio.h | 14 +- lib/librte_eal/linuxapp/eal/eal_vfio.c | 22 +- lib/librte_eal/rte_eal_version.map | 2 + 14 files changed, 441 insertions(+), 38 deletions(-)
Signature · 1 line
2.12.0
PATCH v2 1/6 vfio: allow DMA map of memory for the default vfio fd
Enable users the option to call rte_vfio_dma_map with request to map to the default vfio fd.
--- doc/guides/rel_notes/release_19_05.rst | 3 +++ lib/librte_eal/common/include/rte_vfio.h | 8 ++++++-- lib/librte_eal/linuxapp/eal/eal_vfio.c | 10 ++++++++-- 3 files changed, 17 insertions(+), 4 deletions(-)
doc/guides/rel_notes/release_19_05.rst +3 -0
@@ -110,6 +110,9 @@ ABI Changes Also, make sure to start the actual text at the margin. ========================================================= +* vfio: Functions ``rte_vfio_container_dma_map`` and + ``rte_vfio_container_dma_unmap`` have been extended with an option to + request mapping or un-mapping to the default vfio container fd. Shared Library Versions -----------------------
lib/librte_eal/common/include/rte_vfio.h +6 -2
@@ -73,6 +73,8 @@ struct vfio_info_cap_header {#define RTE_VFIO_CAP_MSIX_MAPPABLE 3#endif +#define RTE_VFIO_DEFAULT_CONTAINER_FD (-1) +#else /* not VFIO_PRESENT */ /* we don't need an actual definition, only pointer is used */
@@ -347,7 +349,8 @@ rte_vfio_container_group_unbind(int container_fd, int iommu_group_num);* Perform DMA mapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. Use RTE_VFIO_DEFAULT_CONTAINER_FD to +* use the default container. * * @param vaddr * Starting virtual address of memory to be mapped.
@@ -370,7 +373,8 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr,* Perform DMA unmapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. Use RTE_VFIO_DEFAULT_CONTAINER_FD to +* use the default container. * * @param vaddr * Starting virtual address of memory to be unmapped.
lib/librte_eal/linuxapp/eal/eal_vfio.c +8 -2
@@ -1897,7 +1897,10 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd ==RTE_VFIO_DEFAULT_CONTAINER_FD ) + vfio_cfg = default_vfio_cfg; +else + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd);if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;
@@ -1917,7 +1920,10 @@ rte_vfio_container_dma_unmap(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd ==RTE_VFIO_DEFAULT_CONTAINER_FD ) + vfio_cfg = default_vfio_cfg; +else + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd);if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;
Signature · 1 line
2.12.0
Re: [PATCH v2 1/6] vfio: allow DMA map of memory for the default vfio fd
Quoted 5 lines · Shahaf Shuler
Enable users the option to call
rte_vfio_dma_mapwith request to map to the default vfio fd.Signed-off-by Shahaf Shuler <[email protected]>---
LGTM
Signature · 2 lines
Thanks, Anatoly
PATCH v2 2/6 vfio: don't fail to DMA map if memory is already mapped
In reply to Shahaf ShulerCurrently vfio DMA map function will fail in case the same memory segment is mapped twice.
This is too strict, as this is not an error to map the same memory twice.
Instead, use the kernel return value to detect such state and have the DMA function to return as successful.
For type1 mapping the kernel driver returns EEXISTS. For spapr mapping EBUSY is returned since kernel 4.10.
--- lib/librte_eal/linuxapp/eal/eal_vfio.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_vfio.c +10 -2
@@ -1263,7 +1263,11 @@ vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,VFIO_DMA_MAP_FLAG_WRITE ; ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map); -if (ret) { +/** +* In case the mapping was already done EEXIST will be +* returned from kernel. +*/ +if (ret && (errno !=EEXIST )) {RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , errno,strerror (errno));return -1 ;
@@ -1324,7 +1328,11 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,VFIO_DMA_MAP_FLAG_WRITE ; ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map); -if (ret) { +/** +* In case the mapping was already done EBUSY will be +* returned from kernel. +*/ +if (ret && (errno !=EBUSY )) {RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , errno,strerror (errno));return -1 ;
Signature · 1 line
2.12.0
Re: [PATCH v2 2/6] vfio: don't fail to DMA map if memory is already mapped
Quoted 38 lines · Shahaf Shuler
Currently vfio DMA map function will fail in case the same memory segment is mapped twice.
This is too strict, as this is not an error to map the same memory twice.
Instead, use the kernel return value to detect such state and have the DMA function to return as successful.
For type1 mapping the kernel driver returns EEXISTS. For spapr mapping EBUSY is returned since kernel 4.10.
Signed-off-by Shahaf Shuler <[email protected]>--- lib/librte_eal/linuxapp/eal/eal_vfio.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal_vfio.c +5
-1@@ -1263,7 +1263,11 @@ vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,VFIO_DMA_MAP_FLAG_WRITE ; ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map); -if (ret) { +/** +* In case the mapping was already done EEXIST will be +* returned from kernel. +*/ +if (ret && (errno !=EEXIST )) {RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , errno,strerror (errno));return -1 ;@@ -1324,7 +1328,11 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,VFIO_DMA_MAP_FLAG_WRITE ;
lib/librte_eal/linuxapp/eal/eal_vfio.c vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova, · line 1338
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_vfio.c +5
-1ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map); -if (ret) { +/** +* In case the mapping was already done EBUSY will be +* returned from kernel. +*/ +if (ret && (errno !=EBUSY )) {RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , errno,strerror (errno));return -1 ;
Nitpicking, but maybe it would be good to throw a debug output about not attempting to map because the area is already mapped. Silently ignoring the error won't help with debugging, should any issues arise :)
Otherwise LGTM
Signature · 2 lines
Thanks, Anatoly
PATCH v2 3/6 bus: introduce device level DMA memory mapping
In reply to Shahaf ShulerThe DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the rte_malloc header).
The user should create a memory, register it through rte_extmem_register API, and call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* call rte_extmem_register on the memory chunk.
* take a device, and query its rte_device.
* call the device specific mapping function for this device.
Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap APIs, leaving the rte device APIs as the preferred option for the user.
--- drivers/bus/pci/pci_common.c | 48 ++++++++++++++++++++++++++++ drivers/bus/pci/rte_bus_pci.h | 40 +++++++++++++++++++++++ lib/librte_eal/common/eal_common_dev.c | 29 +++++++++++++++++ lib/librte_eal/common/include/rte_bus.h | 44 +++++++++++++++++++++++++ lib/librte_eal/common/include/rte_dev.h | 43 +++++++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 2 ++ 6 files changed, 206 insertions(+)
drivers/bus/pci/pci_common.c +48 -0
@@ -528,6 +528,52 @@ pci_unplug(struct rte_device *dev)return ret; } +static int +pci_dma_map (struct rte_device *dev,void *addr,uint64_t iova,size_t len) +{ +struct rte_pci_device *pdev =RTE_DEV_TO_PCI (dev); + +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ; +return -rte_errno; + } +if (pdev->driver->dma_map) +return pdev->driver->dma_map (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_map + (RTE_VFIO_DEFAULT_CONTAINER_FD , (uintptr_t )addr, + iova, len); + rte_errno =ENOTSUP ; +return -rte_errno; +} + +static int +pci_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova,size_t len) +{ +struct rte_pci_device *pdev =RTE_DEV_TO_PCI (dev); + +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ; +return -rte_errno; + } +if (pdev->driver->dma_unmap) +return pdev->driver->dma_unmap (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_unmap + (RTE_VFIO_DEFAULT_CONTAINER_FD , (uintptr_t )addr, + iova, len); + rte_errno =ENOTSUP ; +return -rte_errno; +} +struct rte_pci_bus rte_pci_bus = { .bus = { .scan = rte_pci_scan,
@@ -536,6 +582,8 @@ struct rte_pci_bus rte_pci_bus = { .plug = pci_plug, .unplug = pci_unplug, .parse = pci_parse, + .dma_map = pci_dma_map, + .dma_unmap = pci_dma_unmap, .get_iommu_class = rte_pci_get_iommu_class, .dev_iterate = rte_pci_dev_iterate, .hot_unplug_handler = pci_hot_unplug_handler,
drivers/bus/pci/rte_bus_pci.h +40 -0
@@ -114,6 +114,44 @@ typedef int (pci_probe_t)(struct rte_pci_driver *, struct rte_pci_device *);typedef int (pci_remove_t )(struct rte_pci_device *);/** +* Driver-specific DMA mapping. After a successful call the device +* will be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be mapped. +* @param iova +* Starting IOVA address of memory to be mapped. +* @param len +* Length of memory segment being mapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/ +typedef int (pci_dma_map_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** +* Driver-specific DMA un-mapping. After a successful call the device +* will not be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be unmapped. +* @param iova +* Starting IOVA address of memory to be unmapped. +* @param len +* Length of memory segment being unmapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/ +typedef int (pci_dma_unmap_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver {
@@ -122,6 +160,8 @@ struct rte_pci_driver {struct rte_pci_bus *bus;/**< PCI bus reference. */ pci_probe_t *probe;/**< Device Probe function. */ pci_remove_t *remove;/**< Device Remove function. */ +pci_dma_map_t *dma_map;/**< device dma map function. */ +pci_dma_unmap_t *dma_unmap;/**< device dma unmap function. */ const struct rte_pci_id *id_table;/**< ID table, NULL terminated. */ uint32_t drv_flags;/**< Flags RTE_PCI_DRV_*. */ };
lib/librte_eal/common/eal_common_dev.c +29 -0
@@ -756,3 +756,32 @@ rte_dev_iterator_next(struct rte_dev_iterator *it)free (cls_str);return it->device; } + +int +rte_dev_dma_map (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->dma_map ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -rte_errno; + } +/* Memory must be registered through rte_extmem_* APIs */ +if (rte_mem_virt2memseg (addr,NULL ) ==NULL ) { + rte_errno =EINVAL ; +return -rte_errno; + } + +return dev->bus->dma_map (dev, addr, iova, len); +} + +int +rte_dev_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->dma_unmap ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -rte_errno; + } + +return dev->bus->dma_unmap (dev, addr, iova, len); +}
lib/librte_eal/common/include/rte_bus.h +44 -0
56 lines not on this page · raw message
lib/librte_eal/common/include/rte_dev.h +43 -0
47 lines not on this page · raw message
lib/librte_eal/rte_eal_version.map +2 -0
@@ -277,6 +277,8 @@ EXPERIMENTAL { rte_class_unregister; rte_ctrl_thread_create; rte_delay_us_sleep; + rte_dev_dma_map; + rte_dev_dma_unmap; rte_dev_event_callback_process; rte_dev_event_callback_register; rte_dev_event_callback_unregister;
Signature · 1 line
2.12.0
Re: [PATCH v2 3/6] bus: introduce device level DMA memory mapping
Quoted 44 lines · Shahaf Shuler
The DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the
rte_mallocheader).
The user should create a memory, register it throughrte_extmem_registerAPI, and call DMA map function in order to register such memory to the different devices.The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* callrte_extmem_registeron the memory chunk.
* take a device, and query itsrte_device.
* call the device specific mapping function for this device.Future work will deprecate the
rte_vfio_dma_mapandrte_vfio_dma_unmapAPIs, leaving the rte device APIs as the preferred option for the user.Signed-off-by Shahaf Shuler <[email protected]>---
<snip>
drivers/bus/pci/pci_common.c pci_unplug(struct rte_device *dev) · line 539
Quoted 5 lines
drivers/bus/pci/pci_common.c +5
-0+ +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ; +return -rte_errno; + }
We could put a check in here to see if the memory has been registered with DPDK. Just call rte_mem_virt2memseg_list(addr) - if it returns NULL, the memory wasn't registered, so you can throw an error. Not sure of appropriate errno in that case - ENODEV? EINVAL?
Quoted 10 lines
drivers/bus/pci/pci_common.c +10
-0+if (pdev->driver->dma_map) +return pdev->driver->dma_map (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_map + (RTE_VFIO_DEFAULT_CONTAINER_FD , (uintptr_t )addr, + iova, len);
<snip>
lib/librte_eal/common/eal_common_dev.c rte_dev_iterator_next(struct rte_dev_iterator *it) · line 769
Quoted 9 lines
lib/librte_eal/common/eal_common_dev.c +9
-0+rte_dev_dma_map(struct rte_device *dev, void *addr, uint64_t iova, + size_t len) +{ + if (dev->bus->dma_map == NULL || len == 0) { + rte_errno = EINVAL; +return -rte_errno; + } +/* Memory must be registered through rte_extmem_* APIs */ +if (rte_mem_virt2memseg (addr,NULL ) ==NULL ) {
No need to call rte_mem_virt2memseg - rte_mem_virt2memseg_list will do.
Quoted 5 lines
lib/librte_eal/common/eal_common_dev.c +5
-0+ rte_errno =EINVAL ; +return -rte_errno; + } + +return dev->bus->dma_map (dev, addr, iova, len);
lib/librte_eal/common/eal_common_dev.c rte_dev_iterator_next(struct rte_dev_iterator *it) · line 784
Quoted 10 lines
lib/librte_eal/common/eal_common_dev.c +10
-0+} + +int +rte_dev_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->dma_unmap ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -rte_errno; + }
I think attempting to unmap a memory region that isn't registered should be an error, so rte_mem_virt2memseg_list call should be here too.
Quoted 9 lines
lib/librte_eal/common/eal_common_dev.c +3
-0+ +return dev->bus->dma_unmap (dev, addr, iova, len); +}lib/librte_eal/common/include/rte_bus.h +0
-0@@ -168,6 +168,48 @@ typedef int (*rte_bus_unplug_t)(struct rte_device *dev);typedef int (*rte_bus_parse_t )(const char *name,void *addr);
<snip>
lib/librte_eal/common/include/rte_dev.h rte_dev_hotplug_handle_enable(void); · line 521
Quoted 10 lines
lib/librte_eal/common/include/rte_dev.h +4
-0@@ -515,4 +515,47 @@ rte_dev_hotplug_handle_enable(void); int __rte_experimentalrte_dev_hotplug_handle_disable (void); +/** +* Device level DMA map function. +* After a successful call, the memory segment will be mapped to the +* given device.
here and in unmap:
@note please register memory first
?
lib/librte_eal/common/include/rte_dev.h rte_dev_hotplug_handle_enable(void); · line 534
Quoted 13 lines
lib/librte_eal/common/include/rte_dev.h +13
-0+* +* @param dev +* Device pointer. +* @param addr +* Virtual address to map. +* @param iova +* IOVA address to map. +* @param len +* Length of the memory segment being mapped. +* +* @return +* 0 if mapping was successful. +* Negative value and rte_errno is set otherwise.
Here and in other similar places: why are we setting rte_errno *and* returning -rte_errno? Wouldn't returning -1 be enough?
Signature · 2 lines
Thanks, Anatoly
Re: [PATCH v2 3/6] bus: introduce device level DMA memory mapping
Quoted 48 lines · Anatoly
Quoted 44 lines · Shahaf Shuler
The DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the
rte_mallocheader).
The user should create a memory, register it throughrte_extmem_registerAPI, and call DMA map function in order to register such memory to the different devices.The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* callrte_extmem_registeron the memory chunk.
* take a device, and query itsrte_device.
* call the device specific mapping function for this device.Future work will deprecate the
rte_vfio_dma_mapandrte_vfio_dma_unmapAPIs, leaving the rte device APIs as the preferred option for the user.Signed-off-by Shahaf Shuler <[email protected]>---
<snip>
Quoted 7 lines
Quoted 5 lines
+
+ if (!pdev || !pdev->driver) {
+rte_errno= EINVAL;
+ return -rte_errno;
+ }We could put a check in here to see if the memory has been registered with DPDK. Just call
rte_mem_virt2memseg_list(addr)- if it returns NULL, the memory wasn't registered, so you can throw an error. Not sure of appropriate errno in that case - ENODEV? EINVAL?
Apologies - i meant to delete that, but hit one ctrl+Z too many :(
Signature · 2 lines
Thanks, Anatoly
PATCH v2 4/6 net/mlx5: refactor external memory registration
In reply to Shahaf ShulerMove the memory region creation to a separate function to prepare the ground for the reuse of it on the PCI driver map and unmap functions.
--- drivers/net/mlx5/mlx5_mr.c | 86 +++++++++++++++++++++++++++-------------- 1 file changed, 57 insertions(+), 29 deletions(-)
drivers/net/mlx5/mlx5_mr.c +57 -29
@@ -1109,6 +1109,58 @@ mlx5_mr_flush_local_cache(struct mlx5_mr_ctrl *mr_ctrl) }/** +* Creates a memory region for external memory, that is memory which is not +* part of the DPDK memory segments. +* +* @param dev +* Pointer to the ethernet device. +* @param addr +* Starting virtual address of memory. +* @param len +* Length of memory segment being mapped. +* @param socked_id +* Socket to allocate heap memory for the control structures. +* +* @return +* Pointer to MR structure on success, NULL otherwise. +*/ +static struct mlx5_mr * +mlx5_create_mr_ext (struct rte_eth_dev *dev,uintptr_t addr,size_t len, +int socket_id) +{ +struct priv *priv = dev->data->dev_private; +struct mlx5_mr *mr =NULL ; + + mr =rte_zmalloc_socket (NULL , +RTE_ALIGN_CEIL (sizeof (*mr), +RTE_CACHE_LINE_SIZE ), +RTE_CACHE_LINE_SIZE , socket_id); +if (mr ==NULL ) +return NULL ; + mr->ibv_mr = mlx5_glue->reg_mr (priv->pd, (void *)addr, len, +IBV_ACCESS_LOCAL_WRITE ); +if (mr->ibv_mr ==NULL ) { +DRV_LOG (WARNING , +"port %u fail to create a verbs MR for address (%p)" , + dev->data->port_id, (void *)addr); +rte_free (mr); +return NULL ; + } + mr->msl =NULL ;/* Mark it is external memory. */ + mr->ms_bmp =NULL ; + mr->ms_n =1 ; + mr->ms_bmp_n =1 ; +DRV_LOG (DEBUG , +"port %u MR CREATED (%p) for external memory %p:\n" +" [0x%" PRIxPTR", 0x%" PRIxPTR")," +" lkey=0x%x base_idx=%u ms_n=%u, ms_bmp_n=%u" , + dev->data->port_id, (void *)mr, (void *)addr, + addr, addr + len,rte_cpu_to_be_32 (mr->ibv_mr->lkey), + mr->ms_base_idx, mr->ms_n, mr->ms_bmp_n); +return mr; +} + +/** * Called during rte_mempool_mem_iter() by mlx5_mr_update_ext_mp(). * * Externally allocated chunk is registered and a MR is created for the chunk.
@@ -1142,43 +1194,19 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque,rte_rwlock_read_unlock (&priv->mr.rwlock);if (lkey !=UINT32_MAX )return ; - mr =rte_zmalloc_socket (NULL , -RTE_ALIGN_CEIL (sizeof (*mr), -RTE_CACHE_LINE_SIZE ), -RTE_CACHE_LINE_SIZE , mp->socket_id); -if (mr ==NULL ) { -DRV_LOG (WARNING , -"port %u unable to allocate memory for a new MR of" -" mempool (%s)." , - dev->data->port_id, mp->name); - data->ret =-1 ; -return ; - }DRV_LOG (DEBUG ,"port %u register MR for chunk #%d of mempool (%s)" , dev->data->port_id, mem_idx, mp->name); - mr->ibv_mr = mlx5_glue->reg_mr (priv->pd, (void *)addr, len, -IBV_ACCESS_LOCAL_WRITE ); -if (mr->ibv_mr ==NULL ) { + mr =mlx5_create_mr_ext (dev, addr, len, mp->socket_id); +if (!mr) {DRV_LOG (WARNING , -"port %u fail to create a verbs MR for address (%p)" , - dev->data->port_id, (void *)addr); -rte_free (mr); +"port %u unable to allocate a new MR of" +" mempool (%s)." , + dev->data->port_id, mp->name); data->ret =-1 ;return ; } - mr->msl =NULL ;/* Mark it is external memory. */ - mr->ms_bmp =NULL ; - mr->ms_n =1 ; - mr->ms_bmp_n =1 ;rte_rwlock_write_lock (&priv->mr.rwlock);LIST_INSERT_HEAD (&priv->mr.mr_list, mr, mr); -DRV_LOG (DEBUG , -"port %u MR CREATED (%p) for external memory %p:\n" -" [0x%" PRIxPTR", 0x%" PRIxPTR")," -" lkey=0x%x base_idx=%u ms_n=%u, ms_bmp_n=%u" , - dev->data->port_id, (void *)mr, (void *)addr, - addr, addr + len,rte_cpu_to_be_32 (mr->ibv_mr->lkey), - mr->ms_base_idx, mr->ms_n, mr->ms_bmp_n);/* Insert to the global cache table. */ mr_insert_dev_cache (dev, mr);rte_rwlock_write_unlock (&priv->mr.rwlock);
Signature · 1 line
2.12.0
PATCH v2 5/6 net/mlx5: support PCI device DMA map and unmap
In reply to Shahaf ShulerThe implementation reuses the external memory registration work done by commit[1].
Note about representors:
The current representor design will not work
with those map and unmap functions. The reason is that for representors we have multiple IB devices share the same PCI function, so mapping will happen only on one of the representors and not all of them.
While it is possible to implement such support, the IB representor design is going to be changed during DPDK19.05. The new design will have a single IB device for all representors, hence sharing of a single memory region between all representors will be possible.
[1]
commit 7e43a32ee060
("net/mlx5: support externally allocated static memory")
--- drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 135 ++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 5 ++ 3 files changed, 142 insertions(+)
drivers/net/mlx5/mlx5.c +2 -0
@@ -1626,6 +1626,8 @@ static struct rte_pci_driver mlx5_driver = { .id_table = mlx5_pci_id_map, .probe = mlx5_pci_probe, .remove = mlx5_pci_remove, + .dma_map = mlx5_dma_map, + .dma_unmap = mlx5_dma_unmap, .drv_flags = (RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV | RTE_PCI_DRV_PROBE_AGAIN), };
drivers/net/mlx5/mlx5_mr.c +135 -0
147 lines not on this page · raw message
drivers/net/mlx5/mlx5_rxtx.h +5 -0
@@ -28,6 +28,7 @@#include <rte_atomic.h> #include <rte_spinlock.h> #include <rte_io.h> +#include <rte_bus_pci.h> #include "mlx5_utils.h" #include "mlx5.h"
@@ -367,6 +368,10 @@ uint32_t mlx5_rx_addr2mr_bh(struct mlx5_rxq_data *rxq, uintptr_t addr);uint32_t mlx5_tx_mb2mr_bh (struct mlx5_txq_data *txq,struct rte_mbuf *mb);uint32_t mlx5_tx_update_ext_mp (struct mlx5_txq_data *txq,uintptr_t addr,struct rte_mempool *mp); +int mlx5_dma_map (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len); +int mlx5_dma_unmap (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len);/** * Provide safe 64bit store operation to mlx5 UAR region for both 32bit and
Signature · 1 line
2.12.0
PATCH v2 6/6 doc: deprecate VFIO DMA map APIs
In reply to Shahaf ShulerAs those have been replaced by rte_bus_dma_map and rte_pci_dma_unmap APIs.
--- doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 ++++ lib/librte_eal/common/include/rte_vfio.h | 6 ++++-- 3 files changed, 9 insertions(+), 3 deletions(-)
doc/guides/prog_guide/env_abstraction_layer.rst +1 -1
@@ -282,7 +282,7 @@ The expected workflow is as follows: - If IOVA table is not specified, IOVA addresses will be assumed to be unavailable - Other processes must attach to the memory area before they can use it -* Perform DMA mapping with ``rte_vfio_dma_map`` if needed +* Perform DMA mapping with ``rte_bus_dma_map`` if needed * Use the memory area in your application * If memory area is no longer needed, it can be unregistered - If the area was mapped for DMA, unmapping must be performed before
doc/guides/rel_notes/deprecation.rst +4 -0
@@ -35,6 +35,10 @@ Deprecation Notices + ``rte_eal_devargs_type_count`` +* vfio: removal of ``rte_vfio_dma_map`` and ``rte_vfio_dma_unmap`` APIs which + have been replaced with ``rte_bus_dma_map`` and ``rte_bus_dma_unmap`` + functions. The due date for the removal targets DPDK 19.08. + * pci: Several exposed functions are misnamed. The following functions are deprecated starting from v17.11 and are replaced:
lib/librte_eal/common/include/rte_vfio.h +4 -2
@@ -190,6 +190,7 @@ int rte_vfio_clear_group(int vfio_group_fd);/** +* @deprecated * Map memory region for use with VFIO. * * @note Require at least one device to be attached at the time of
@@ -210,11 +211,12 @@ rte_vfio_clear_group(int vfio_group_fd);* 0 if success. * -1 on error. */ -int +int __rte_deprecatedrte_vfio_dma_map (uint64_t vaddr,uint64_t iova,uint64_t len);/** +* @deprecated * Unmap memory region from VFIO. * * @param vaddr
@@ -231,7 +233,7 @@ rte_vfio_dma_map(uint64_t vaddr, uint64_t iova, uint64_t len);* -1 on error. */ -int +int __rte_deprecatedrte_vfio_dma_unmap (uint64_t vaddr,uint64_t iova,uint64_t len);/** * Parse IOMMU group number for a device
Signature · 1 line
2.12.0
Re: [PATCH v2 6/6] doc: deprecate VFIO DMA map APIs
Quoted 79 lines · Shahaf Shuler
As those have been replaced by
rte_bus_dma_mapandrte_pci_dma_unmapAPIs.Signed-off-by Shahaf Shuler <[email protected]>--- doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 ++++ lib/librte_eal/common/include/rte_vfio.h | 6 ++++-- 3 files changed, 9 insertions(+), 3 deletions(-)doc/guides/prog_guide/env_abstraction_layer.rst +0
-0@@ -282,7 +282,7 @@ The expected workflow is as follows: - If IOVA table is not specified, IOVA addresses will be assumed to be unavailable - Other processes must attach to the memory area before they can useit
doc/guides/prog_guide/env_abstraction_layer.rst +1
-1-* Perform DMA mapping with ``rte_vfio_dma_map`` if needed +* Perform DMA mapping with ``rte_bus_dma_map`` if needed * Use the memory area in your application * If memory area is no longer needed, it can be unregistered - If the area was mapped for DMA, unmapping must be performed beforedoc/guides/rel_notes/deprecation.rst +1
-0@@ -35,6 +35,10 @@ Deprecation Notices + ``rte_eal_devargs_type_count`` +* vfio: removal of ``rte_vfio_dma_map`` and ``rte_vfio_dma_unmap`` APIswhich
doc/guides/rel_notes/deprecation.rst +3
-0+ have been replaced with ``rte_bus_dma_map`` and ``rte_bus_dma_unmap`` + functions. The due date for the removal targets DPDK 19.08. + * pci: Several exposed functions are misnamed. The following functions are deprecated starting from v17.11 and arereplaced:
lib/librte_eal/common/include/rte_vfio.h +4
-2@@ -190,6 +190,7 @@ int rte_vfio_clear_group(int vfio_group_fd);/** +* @deprecated * Map memory region for use with VFIO. * * @note Require at least one device to be attached at the time of @@ -210,11 +211,12 @@ rte_vfio_clear_group(int vfio_group_fd);* 0 if success. * -1 on error. */ -int +int __rte_deprecatedrte_vfio_dma_map (uint64_t vaddr,uint64_t iova,uint64_t len);/** +* @deprecated * Unmap memory region from VFIO. * * @param vaddr @@ -231,7 +233,7 @@ rte_vfio_dma_map(uint64_t vaddr, uint64_t iova,uint64_t len); * -1 on error. */ -int +int __rte_deprecatedrte_vfio_dma_unmap (uint64_t vaddr,uint64_t iova,uint64_t len);/**
Quoted 5 lines
* Parse IOMMU group number for a device
--
2.12.0
I don't know users of such apis, but you can't mark rte_vfio_dma_map as deprecated now.
There is no stable alternative, rte_bus_dma_map() is experimental.
This has been discussed and some patches are in progress about it. Last version: http://patchwork.dpdk.org/patch/50040/
Signature · 1 line
David Marchand
v3 cover introduce DMA memory mapping for external memory
In reply to Shahaf ShulerThe DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the rte_malloc header).
The user should create a memory, register it through rte_extmem_register API, and call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* call rte_extmem_register on the memory chunk.
* take a device, and query its rte_device.
* call the device specific mapping function for this device.
Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap APIs, leaving the rte device APIs as the preferred option for the user.
On v3:
- Fixed compilation issue on freebsd.
- Fixed forgotten rte_bus_dma_map to rte_dev_dma_map.
- Removed __rte_deprecated from vfio function till the time the rte_dev_dma_map will be non-experimental.
- Changed error return value to always be -1, with proper errno.
- Used rte_mem_virt2memseg_list instead of rte_mem_virt2memseg to verify memory is registered.
- Added above check also on dma_unmap calls.
- Added note in the API the memory must be registered in advance.
- Added debug log to report the case memory mapping to vfio was skipped.
On v2:
- Added warn in release notes about the API change in vfio.
- Moved function doc to prototype declaration.
- Used dma_map and dma_unmap instead of map and unmap.
- Used RTE_VFIO_DEFAULT_CONTAINER_FD instead of -1 fixed value.
- Moved bus function to eal_common_dev.c and renamed them properly.
- Changed eth device iterator to use RTE_DEV_FOREACH.
- Enforced memory is registered with rte_extmem_* prior to mapping.
- Used EEXIST as the only possible return value from type1 vfio IOMMU mapping.
[1] https://patches.dpdk.org/patch/47796/
Shahaf Shuler (6):
vfio: allow DMA map of memory for the default vfio fd
vfio: don't fail to DMA map if memory is already mapped
bus: introduce device level DMA memory mapping net/mlx5: refactor external memory registration net/mlx5: support PCI device DMA map and unmap
doc: deprecation notice for VFIO DMA map APIs
doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 + doc/guides/rel_notes/release_19_05.rst | 3 + drivers/bus/pci/pci_common.c | 48 ++++ drivers/bus/pci/rte_bus_pci.h | 40 ++++ drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 225 ++++++++++++++++--- drivers/net/mlx5/mlx5_rxtx.h | 5 + lib/librte_eal/common/eal_common_dev.c | 34 +++ lib/librte_eal/common/include/rte_bus.h | 44 ++++ lib/librte_eal/common/include/rte_dev.h | 47 ++++ lib/librte_eal/common/include/rte_vfio.h | 8 +- lib/librte_eal/linuxapp/eal/eal_vfio.c | 42 +++- lib/librte_eal/rte_eal_version.map | 2 + 14 files changed, 468 insertions(+), 38 deletions(-)
Signature · 1 line
2.12.0
PATCH v3 1/6 vfio: allow DMA map of memory for the default vfio fd
Enable users the option to call rte_vfio_dma_map with request to map to the default vfio fd.
--- doc/guides/rel_notes/release_19_05.rst | 3 +++ lib/librte_eal/common/include/rte_vfio.h | 8 ++++++-- lib/librte_eal/linuxapp/eal/eal_vfio.c | 10 ++++++++-- 3 files changed, 17 insertions(+), 4 deletions(-)
doc/guides/rel_notes/release_19_05.rst +3 -0
@@ -122,6 +122,9 @@ ABI Changes Also, make sure to start the actual text at the margin. ========================================================= +* vfio: Functions ``rte_vfio_container_dma_map`` and + ``rte_vfio_container_dma_unmap`` have been extended with an option to + request mapping or un-mapping to the default vfio container fd. Shared Library Versions -----------------------
lib/librte_eal/common/include/rte_vfio.h +6 -2
@@ -80,6 +80,8 @@ struct vfio_device_info;#endif /* VFIO_PRESENT */ +#define RTE_VFIO_DEFAULT_CONTAINER_FD (-1) +/** * Setup vfio_cfg for the device identified by its address. * It discovers the configured I/O MMU groups or sets a new one for the device.
@@ -347,7 +349,8 @@ rte_vfio_container_group_unbind(int container_fd, int iommu_group_num);* Perform DMA mapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. Use RTE_VFIO_DEFAULT_CONTAINER_FD to +* use the default container. * * @param vaddr * Starting virtual address of memory to be mapped.
@@ -370,7 +373,8 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr,* Perform DMA unmapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. Use RTE_VFIO_DEFAULT_CONTAINER_FD to +* use the default container. * * @param vaddr * Starting virtual address of memory to be unmapped.
lib/librte_eal/linuxapp/eal/eal_vfio.c +8 -2
@@ -1897,7 +1897,10 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd ==RTE_VFIO_DEFAULT_CONTAINER_FD ) + vfio_cfg = default_vfio_cfg; +else + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd);if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;
@@ -1917,7 +1920,10 @@ rte_vfio_container_dma_unmap(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd ==RTE_VFIO_DEFAULT_CONTAINER_FD ) + vfio_cfg = default_vfio_cfg; +else + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd);if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;
Signature · 1 line
2.12.0
PATCH v3 2/6 vfio: don't fail to DMA map if memory is already mapped
In reply to Shahaf ShulerCurrently vfio DMA map function will fail in case the same memory segment is mapped twice.
This is too strict, as this is not an error to map the same memory twice.
Instead, use the kernel return value to detect such state and have the DMA function to return as successful.
For type1 mapping the kernel driver returns EEXISTS. For spapr mapping EBUSY is returned since kernel 4.10.
--- lib/librte_eal/linuxapp/eal/eal_vfio.c | 32 +++++++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-)
lib/librte_eal/linuxapp/eal/eal_vfio.c +28 -4
@@ -1264,9 +1264,21 @@ vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova, ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map);if (ret) { -RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , - errno,strerror (errno)); +/** +* In case the mapping was already done EEXIST will be +* returned from kernel. +*/ +if (errno ==EEXIST ) { +RTE_LOG (DEBUG ,EAL , +" Memory segment is allready mapped," +" skipping" ); + }else { +RTE_LOG (ERR ,EAL , +" cannot set up DMA remapping," +" error %i (%s)\n" , + errno,strerror (errno));return -1 ; + } } }else {memset (&dma_unmap,0 ,sizeof (dma_unmap));
@@ -1325,9 +1337,21 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova, ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map);if (ret) { -RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , - errno,strerror (errno)); +/** +* In case the mapping was already done EBUSY will be +* returned from kernel. +*/ +if (errno ==EBUSY ) { +RTE_LOG (DEBUG ,EAL , +" Memory segment is allready mapped," +" skipping" ); + }else { +RTE_LOG (ERR ,EAL , +" cannot set up DMA remapping," +" error %i (%s)\n" , errno, +strerror (errno));return -1 ; + } } }else {
Signature · 1 line
2.12.0
PATCH v3 3/6 bus: introduce device level DMA memory mapping
In reply to Shahaf ShulerThe DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the rte_malloc header).
The user should create a memory, register it through rte_extmem_register API, and call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* call rte_extmem_register on the memory chunk.
* take a device, and query its rte_device.
* call the device specific mapping function for this device.
Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap APIs, leaving the rte device APIs as the preferred option for the user.
--- drivers/bus/pci/pci_common.c | 48 ++++++++++++++++++++++++++++ drivers/bus/pci/rte_bus_pci.h | 40 +++++++++++++++++++++++ lib/librte_eal/common/eal_common_dev.c | 34 ++++++++++++++++++++ lib/librte_eal/common/include/rte_bus.h | 44 +++++++++++++++++++++++++ lib/librte_eal/common/include/rte_dev.h | 47 +++++++++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 2 ++ 6 files changed, 215 insertions(+)
drivers/bus/pci/pci_common.c +48 -0
@@ -528,6 +528,52 @@ pci_unplug(struct rte_device *dev)return ret; } +static int +pci_dma_map (struct rte_device *dev,void *addr,uint64_t iova,size_t len) +{ +struct rte_pci_device *pdev =RTE_DEV_TO_PCI (dev); + +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ; +return -1 ; + } +if (pdev->driver->dma_map) +return pdev->driver->dma_map (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_map + (RTE_VFIO_DEFAULT_CONTAINER_FD , (uintptr_t )addr, + iova, len); + rte_errno =ENOTSUP ; +return -1 ; +} + +static int +pci_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova,size_t len) +{ +struct rte_pci_device *pdev =RTE_DEV_TO_PCI (dev); + +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ; +return -1 ; + } +if (pdev->driver->dma_unmap) +return pdev->driver->dma_unmap (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_unmap + (RTE_VFIO_DEFAULT_CONTAINER_FD , (uintptr_t )addr, + iova, len); + rte_errno =ENOTSUP ; +return -1 ; +} +struct rte_pci_bus rte_pci_bus = { .bus = { .scan = rte_pci_scan,
@@ -536,6 +582,8 @@ struct rte_pci_bus rte_pci_bus = { .plug = pci_plug, .unplug = pci_unplug, .parse = pci_parse, + .dma_map = pci_dma_map, + .dma_unmap = pci_dma_unmap, .get_iommu_class = rte_pci_get_iommu_class, .dev_iterate = rte_pci_dev_iterate, .hot_unplug_handler = pci_hot_unplug_handler,
drivers/bus/pci/rte_bus_pci.h +40 -0
@@ -114,6 +114,44 @@ typedef int (pci_probe_t)(struct rte_pci_driver *, struct rte_pci_device *);typedef int (pci_remove_t )(struct rte_pci_device *);/** +* Driver-specific DMA mapping. After a successful call the device +* will be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be mapped. +* @param iova +* Starting IOVA address of memory to be mapped. +* @param len +* Length of memory segment being mapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/ +typedef int (pci_dma_map_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** +* Driver-specific DMA un-mapping. After a successful call the device +* will not be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be unmapped. +* @param iova +* Starting IOVA address of memory to be unmapped. +* @param len +* Length of memory segment being unmapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/ +typedef int (pci_dma_unmap_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver {
@@ -122,6 +160,8 @@ struct rte_pci_driver {struct rte_pci_bus *bus;/**< PCI bus reference. */ pci_probe_t *probe;/**< Device Probe function. */ pci_remove_t *remove;/**< Device Remove function. */ +pci_dma_map_t *dma_map;/**< device dma map function. */ +pci_dma_unmap_t *dma_unmap;/**< device dma unmap function. */ const struct rte_pci_id *id_table;/**< ID table, NULL terminated. */ uint32_t drv_flags;/**< Flags RTE_PCI_DRV_*. */ };
lib/librte_eal/common/eal_common_dev.c +34 -0
@@ -756,3 +756,37 @@ rte_dev_iterator_next(struct rte_dev_iterator *it)free (cls_str);return it->device; } + +int +rte_dev_dma_map (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->dma_map ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -1 ; + } +/* Memory must be registered through rte_extmem_* APIs */ +if (rte_mem_virt2memseg_list (addr) ==NULL ) { + rte_errno =EINVAL ; +return -1 ; + } + +return dev->bus->dma_map (dev, addr, iova, len); +} + +int +rte_dev_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->dma_unmap ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -1 ; + } +/* Memory must be registered through rte_extmem_* APIs */ +if (rte_mem_virt2memseg_list (addr) ==NULL ) { + rte_errno =EINVAL ; +return -1 ; + } + +return dev->bus->dma_unmap (dev, addr, iova, len); +}
lib/librte_eal/common/include/rte_bus.h +44 -0
56 lines not on this page · raw message
lib/librte_eal/common/include/rte_dev.h +47 -0
51 lines not on this page · raw message
lib/librte_eal/rte_eal_version.map +2 -0
@@ -277,6 +277,8 @@ EXPERIMENTAL { rte_class_unregister; rte_ctrl_thread_create; rte_delay_us_sleep; + rte_dev_dma_map; + rte_dev_dma_unmap; rte_dev_event_callback_process; rte_dev_event_callback_register; rte_dev_event_callback_unregister;
Signature · 1 line
2.12.0
Re: [PATCH v3 3/6] bus: introduce device level DMA memory mapping
Quoted 44 lines · Shahaf Shuler
The DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the
rte_mallocheader).
The user should create a memory, register it throughrte_extmem_registerAPI, and call DMA map function in order to register such memory to the different devices.The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* callrte_extmem_registeron the memory chunk.
* take a device, and query itsrte_device.
* call the device specific mapping function for this device.Future work will deprecate the
rte_vfio_dma_mapandrte_vfio_dma_unmapAPIs, leaving the rte device APIs as the preferred option for the user.Signed-off-by Shahaf Shuler <[email protected]>---
<snip>
Quoted 6 lines
lib/librte_eal/common/eal_common_dev.c +0
-0@@ -756,3 +756,37 @@ rte_dev_iterator_next(struct rte_dev_iterator *it)free (cls_str);
lib/librte_eal/common/eal_common_dev.c rte_dev_iterator_next(struct rte_dev_iterator *it) · line 766
Quoted 10 lines
lib/librte_eal/common/eal_common_dev.c +8
-0return it->device; } + +int +rte_dev_dma_map (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->dma_map ==NULL || len ==0 ) { + rte_errno =EINVAL ; +return -1 ;
Here and below as well - please correct me if i'm wrong here, but should not having dma_map defined for a bus be an EINVAL error? EINVAL is reserved for invalid arguments - this seems to me that ENOTSUP would be more appropriate error code. User should be able to differentiate between failed mapping due to invalid data, and failed data due to the device's bus not supporting DMA remapping API in the first place.
Signature · 2 lines
Thanks, Anatoly
PATCH v3 4/6 net/mlx5: refactor external memory registration
In reply to Shahaf ShulerMove the memory region creation to a separate function to prepare the ground for the reuse of it on the PCI driver map and unmap functions.
--- drivers/net/mlx5/mlx5_mr.c | 86 +++++++++++++++++++++++++++-------------- 1 file changed, 57 insertions(+), 29 deletions(-)
drivers/net/mlx5/mlx5_mr.c +57 -29
@@ -1109,6 +1109,58 @@ mlx5_mr_flush_local_cache(struct mlx5_mr_ctrl *mr_ctrl) }/** +* Creates a memory region for external memory, that is memory which is not +* part of the DPDK memory segments. +* +* @param dev +* Pointer to the ethernet device. +* @param addr +* Starting virtual address of memory. +* @param len +* Length of memory segment being mapped. +* @param socked_id +* Socket to allocate heap memory for the control structures. +* +* @return +* Pointer to MR structure on success, NULL otherwise. +*/ +static struct mlx5_mr * +mlx5_create_mr_ext (struct rte_eth_dev *dev,uintptr_t addr,size_t len, +int socket_id) +{ +struct mlx5_priv *priv = dev->data->dev_private; +struct mlx5_mr *mr =NULL ; + + mr =rte_zmalloc_socket (NULL , +RTE_ALIGN_CEIL (sizeof (*mr), +RTE_CACHE_LINE_SIZE ), +RTE_CACHE_LINE_SIZE , socket_id); +if (mr ==NULL ) +return NULL ; + mr->ibv_mr = mlx5_glue->reg_mr (priv->pd, (void *)addr, len, +IBV_ACCESS_LOCAL_WRITE ); +if (mr->ibv_mr ==NULL ) { +DRV_LOG (WARNING , +"port %u fail to create a verbs MR for address (%p)" , + dev->data->port_id, (void *)addr); +rte_free (mr); +return NULL ; + } + mr->msl =NULL ;/* Mark it is external memory. */ + mr->ms_bmp =NULL ; + mr->ms_n =1 ; + mr->ms_bmp_n =1 ; +DRV_LOG (DEBUG , +"port %u MR CREATED (%p) for external memory %p:\n" +" [0x%" PRIxPTR", 0x%" PRIxPTR")," +" lkey=0x%x base_idx=%u ms_n=%u, ms_bmp_n=%u" , + dev->data->port_id, (void *)mr, (void *)addr, + addr, addr + len,rte_cpu_to_be_32 (mr->ibv_mr->lkey), + mr->ms_base_idx, mr->ms_n, mr->ms_bmp_n); +return mr; +} + +/** * Called during rte_mempool_mem_iter() by mlx5_mr_update_ext_mp(). * * Externally allocated chunk is registered and a MR is created for the chunk.
@@ -1142,43 +1194,19 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque,rte_rwlock_read_unlock (&priv->mr.rwlock);if (lkey !=UINT32_MAX )return ; - mr =rte_zmalloc_socket (NULL , -RTE_ALIGN_CEIL (sizeof (*mr), -RTE_CACHE_LINE_SIZE ), -RTE_CACHE_LINE_SIZE , mp->socket_id); -if (mr ==NULL ) { -DRV_LOG (WARNING , -"port %u unable to allocate memory for a new MR of" -" mempool (%s)." , - dev->data->port_id, mp->name); - data->ret =-1 ; -return ; - }DRV_LOG (DEBUG ,"port %u register MR for chunk #%d of mempool (%s)" , dev->data->port_id, mem_idx, mp->name); - mr->ibv_mr = mlx5_glue->reg_mr (priv->pd, (void *)addr, len, -IBV_ACCESS_LOCAL_WRITE ); -if (mr->ibv_mr ==NULL ) { + mr =mlx5_create_mr_ext (dev, addr, len, mp->socket_id); +if (!mr) {DRV_LOG (WARNING , -"port %u fail to create a verbs MR for address (%p)" , - dev->data->port_id, (void *)addr); -rte_free (mr); +"port %u unable to allocate a new MR of" +" mempool (%s)." , + dev->data->port_id, mp->name); data->ret =-1 ;return ; } - mr->msl =NULL ;/* Mark it is external memory. */ - mr->ms_bmp =NULL ; - mr->ms_n =1 ; - mr->ms_bmp_n =1 ;rte_rwlock_write_lock (&priv->mr.rwlock);LIST_INSERT_HEAD (&priv->mr.mr_list, mr, mr); -DRV_LOG (DEBUG , -"port %u MR CREATED (%p) for external memory %p:\n" -" [0x%" PRIxPTR", 0x%" PRIxPTR")," -" lkey=0x%x base_idx=%u ms_n=%u, ms_bmp_n=%u" , - dev->data->port_id, (void *)mr, (void *)addr, - addr, addr + len,rte_cpu_to_be_32 (mr->ibv_mr->lkey), - mr->ms_base_idx, mr->ms_n, mr->ms_bmp_n);/* Insert to the global cache table. */ mr_insert_dev_cache (dev, mr);rte_rwlock_write_unlock (&priv->mr.rwlock);
Signature · 1 line
2.12.0
PATCH v3 5/6 net/mlx5: support PCI device DMA map and unmap
In reply to Shahaf ShulerThe implementation reuses the external memory registration work done by commit[1].
Note about representors:
The current representor design will not work
with those map and unmap functions. The reason is that for representors we have multiple IB devices share the same PCI function, so mapping will happen only on one of the representors and not all of them.
While it is possible to implement such support, the IB representor design is going to be changed during DPDK19.05. The new design will have a single IB device for all representors, hence sharing of a single memory region between all representors will be possible.
[1]
commit 7e43a32ee060
("net/mlx5: support externally allocated static memory")
--- drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 139 ++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 5 ++ 3 files changed, 146 insertions(+)
drivers/net/mlx5/mlx5.c +2 -0
@@ -1630,6 +1630,8 @@ static struct rte_pci_driver mlx5_driver = { .id_table = mlx5_pci_id_map, .probe = mlx5_pci_probe, .remove = mlx5_pci_remove, + .dma_map = mlx5_dma_map, + .dma_unmap = mlx5_dma_unmap, .drv_flags = (RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV | RTE_PCI_DRV_PROBE_AGAIN), };
drivers/net/mlx5/mlx5_mr.c +139 -0
151 lines not on this page · raw message
drivers/net/mlx5/mlx5_rxtx.h +5 -0
@@ -28,6 +28,7 @@#include <rte_atomic.h> #include <rte_spinlock.h> #include <rte_io.h> +#include <rte_bus_pci.h> #include "mlx5_utils.h" #include "mlx5.h"
@@ -367,6 +368,10 @@ uint32_t mlx5_rx_addr2mr_bh(struct mlx5_rxq_data *rxq, uintptr_t addr);uint32_t mlx5_tx_mb2mr_bh (struct mlx5_txq_data *txq,struct rte_mbuf *mb);uint32_t mlx5_tx_update_ext_mp (struct mlx5_txq_data *txq,uintptr_t addr,struct rte_mempool *mp); +int mlx5_dma_map (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len); +int mlx5_dma_unmap (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len);/** * Provide safe 64bit store operation to mlx5 UAR region for both 32bit and
Signature · 1 line
2.12.0
PATCH v3 6/6 doc: deprecation notice for VFIO DMA map APIs
In reply to Shahaf ShulerAs those should be replaced by rte_dev_dma_map and rte_dev_dma_unmap APIs.
--- doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-)
doc/guides/prog_guide/env_abstraction_layer.rst +1 -1
@@ -282,7 +282,7 @@ The expected workflow is as follows: - If IOVA table is not specified, IOVA addresses will be assumed to be unavailable - Other processes must attach to the memory area before they can use it -* Perform DMA mapping with ``rte_vfio_dma_map`` if needed +* Perform DMA mapping with ``rte_dev_dma_map`` if needed * Use the memory area in your application * If memory area is no longer needed, it can be unregistered - If the area was mapped for DMA, unmapping must be performed before
doc/guides/rel_notes/deprecation.rst +4 -0
@@ -35,6 +35,10 @@ Deprecation Notices + ``rte_eal_devargs_type_count`` +* vfio: removal of ``rte_vfio_dma_map`` and ``rte_vfio_dma_unmap`` APIs which + have been replaced with ``rte_dev_dma_map`` and ``rte_dev_dma_unmap`` + functions. The due date for the removal targets DPDK 20.02. + * pci: Several exposed functions are misnamed. The following functions are deprecated starting from v17.11 and are replaced:
Signature · 1 line
2.12.0
v4 cover introduce DMA memory mapping for external memory
In reply to Shahaf ShulerThe DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the rte_malloc header).
The user should create a memory, register it through rte_extmem_register API, and call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* call rte_extmem_register on the memory chunk.
* take a device, and query its rte_device.
* call the device specific mapping function for this device.
Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap APIs, leaving the rte device APIs as the preferred option for the user.
On v4:
- Changed rte_dev_dma_map errno to ENOTSUP in case bus doesn't support DMA map API.
On v3:
- Fixed compilation issue on freebsd.
- Fixed forgotten rte_bus_dma_map to rte_dev_dma_map.
- Removed __rte_deprecated from vfio function till the time the rte_dev_dma_map will be non-experimental.
- Changed error return value to always be -1, with proper errno.
- Used rte_mem_virt2memseg_list instead of rte_mem_virt2memseg to verify memory is registered.
- Added above check also on dma_unmap calls.
- Added note in the API the memory must be registered in advance.
- Added debug log to report the case memory mapping to vfio was skipped.
On v2:
- Added warn in release notes about the API change in vfio.
- Moved function doc to prototype declaration.
- Used dma_map and dma_unmap instead of map and unmap.
- Used RTE_VFIO_DEFAULT_CONTAINER_FD instead of -1 fixed value.
- Moved bus function to eal_common_dev.c and renamed them properly.
- Changed eth device iterator to use RTE_DEV_FOREACH.
- Enforced memory is registered with rte_extmem_* prior to mapping.
- Used EEXIST as the only possible return value from type1 vfio IOMMU mapping.
[1] https://patches.dpdk.org/patch/47796/
Shahaf Shuler (6):
vfio: allow DMA map of memory for the default vfio fd
vfio: don't fail to DMA map if memory is already mapped
bus: introduce device level DMA memory mapping net/mlx5: refactor external memory registration net/mlx5: support PCI device DMA map and unmap
doc: deprecation notice for VFIO DMA map APIs
doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 + doc/guides/rel_notes/release_19_05.rst | 3 + drivers/bus/pci/pci_common.c | 48 ++++ drivers/bus/pci/rte_bus_pci.h | 40 ++++ drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 225 ++++++++++++++++--- drivers/net/mlx5/mlx5_rxtx.h | 5 + lib/librte_eal/common/eal_common_dev.c | 34 +++ lib/librte_eal/common/include/rte_bus.h | 44 ++++ lib/librte_eal/common/include/rte_dev.h | 47 ++++ lib/librte_eal/common/include/rte_vfio.h | 8 +- lib/librte_eal/linuxapp/eal/eal_vfio.c | 42 +++- lib/librte_eal/rte_eal_version.map | 2 + 14 files changed, 468 insertions(+), 38 deletions(-)
Signature · 1 line
2.12.0
PATCH v4 1/6 vfio: allow DMA map of memory for the default vfio fd
Enable users the option to call rte_vfio_dma_map with request to map to the default vfio fd.
--- doc/guides/rel_notes/release_19_05.rst | 3 +++ lib/librte_eal/common/include/rte_vfio.h | 8 ++++++-- lib/librte_eal/linuxapp/eal/eal_vfio.c | 10 ++++++++-- 3 files changed, 17 insertions(+), 4 deletions(-)
doc/guides/rel_notes/release_19_05.rst +3 -0
@@ -136,6 +136,9 @@ ABI Changes Also, make sure to start the actual text at the margin. ========================================================= +* vfio: Functions ``rte_vfio_container_dma_map`` and + ``rte_vfio_container_dma_unmap`` have been extended with an option to + request mapping or un-mapping to the default vfio container fd. Shared Library Versions -----------------------
lib/librte_eal/common/include/rte_vfio.h +6 -2
@@ -80,6 +80,8 @@ struct vfio_device_info;#endif /* VFIO_PRESENT */ +#define RTE_VFIO_DEFAULT_CONTAINER_FD (-1) +/** * Setup vfio_cfg for the device identified by its address. * It discovers the configured I/O MMU groups or sets a new one for the device.
@@ -347,7 +349,8 @@ rte_vfio_container_group_unbind(int container_fd, int iommu_group_num);* Perform DMA mapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. Use RTE_VFIO_DEFAULT_CONTAINER_FD to +* use the default container. * * @param vaddr * Starting virtual address of memory to be mapped.
@@ -370,7 +373,8 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr,* Perform DMA unmapping for devices in a container. * * @param container_fd -* the specified container fd +* the specified container fd. Use RTE_VFIO_DEFAULT_CONTAINER_FD to +* use the default container. * * @param vaddr * Starting virtual address of memory to be unmapped.
lib/librte_eal/linuxapp/eal/eal_vfio.c +8 -2
@@ -1897,7 +1897,10 @@ rte_vfio_container_dma_map(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd ==RTE_VFIO_DEFAULT_CONTAINER_FD ) + vfio_cfg = default_vfio_cfg; +else + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd);if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;
@@ -1917,7 +1920,10 @@ rte_vfio_container_dma_unmap(int container_fd, uint64_t vaddr, uint64_t iova,return -1 ; } - vfio_cfg =get_vfio_cfg_by_container_fd (container_fd); +if (container_fd ==RTE_VFIO_DEFAULT_CONTAINER_FD ) + vfio_cfg = default_vfio_cfg; +else + vfio_cfg =get_vfio_cfg_by_container_fd (container_fd);if (vfio_cfg ==NULL ) {RTE_LOG (ERR ,EAL ,"Invalid container fd\n" );return -1 ;
Signature · 1 line
2.12.0
Re: [PATCH v4 1/6] vfio: allow DMA map of memory for the default vfio fd
10/03/2019 09:27, Shahaf Shuler:
doc/guides/rel_notes/release_19_05.rst ABI Changes · line 138
Quoted 12 lines
Enable users the option to call
rte_vfio_dma_mapwith request to map to the default vfio fd.Signed-off-by Shahaf Shuler <[email protected]>Acked-by Anatoly Burakov <[email protected]>---
doc/guides/rel_notes/release_19_05.rst +3
-0@@ -136,6 +136,9 @@ ABI Changes +* vfio: Functions ``rte_vfio_container_dma_map`` and + ``rte_vfio_container_dma_unmap`` have been extended with an option to + request mapping or un-mapping to the default vfio container fd.
Isn't it an API change rather than ABI?
It is adding -1 as a special value, I think it is not breaking previous interface, and does not require a notification in the release notes.
Re: [PATCH v4 1/6] vfio: allow DMA map of memory for the default vfio fd
30/03/2019 01:23, Thomas Monjalon:
Quoted 4 lines
10/03/2019 09:27, Shahaf Shuler:
Quoted 3 lines
Enable users the option to call
rte_vfio_dma_mapwith request to map to the default vfio fd.
Quoted 13 lines
doc/guides/rel_notes/release_19_05.rst ABI Changes · line 138
Quoted 9 lines
Signed-off-by Shahaf Shuler <[email protected]>Acked-by Anatoly Burakov <[email protected]>---
doc/guides/rel_notes/release_19_05.rst +3
-0@@ -136,6 +136,9 @@ ABI Changes +* vfio: Functions ``rte_vfio_container_dma_map`` and + ``rte_vfio_container_dma_unmap`` have been extended with an option to + request mapping or un-mapping to the default vfio container fd.Isn't it an API change rather than ABI?
It is adding -1 as a special value, I think it is not breaking previous interface, and does not require a notification in the release notes.
I will move it to the "API changes" section.
PATCH v4 2/6 vfio: don't fail to DMA map if memory is already mapped
In reply to Shahaf ShulerCurrently vfio DMA map function will fail in case the same memory segment is mapped twice.
This is too strict, as this is not an error to map the same memory twice.
Instead, use the kernel return value to detect such state and have the DMA function to return as successful.
For type1 mapping the kernel driver returns EEXISTS. For spapr mapping EBUSY is returned since kernel 4.10.
--- lib/librte_eal/linuxapp/eal/eal_vfio.c | 32 +++++++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-)
lib/librte_eal/linuxapp/eal/eal_vfio.c +28 -4
@@ -1264,9 +1264,21 @@ vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova, ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map);if (ret) { -RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , - errno,strerror (errno)); +/** +* In case the mapping was already done EEXIST will be +* returned from kernel. +*/ +if (errno ==EEXIST ) { +RTE_LOG (DEBUG ,EAL , +" Memory segment is allready mapped," +" skipping" ); + }else { +RTE_LOG (ERR ,EAL , +" cannot set up DMA remapping," +" error %i (%s)\n" , + errno,strerror (errno));return -1 ; + } } }else {memset (&dma_unmap,0 ,sizeof (dma_unmap));
@@ -1325,9 +1337,21 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova, ret =ioctl (vfio_container_fd,VFIO_IOMMU_MAP_DMA , &dma_map);if (ret) { -RTE_LOG (ERR ,EAL ," cannot set up DMA remapping, error %i (%s)\n" , - errno,strerror (errno)); +/** +* In case the mapping was already done EBUSY will be +* returned from kernel. +*/ +if (errno ==EBUSY ) { +RTE_LOG (DEBUG ,EAL , +" Memory segment is allready mapped," +" skipping" ); + }else { +RTE_LOG (ERR ,EAL , +" cannot set up DMA remapping," +" error %i (%s)\n" , errno, +strerror (errno));return -1 ; + } } }else {
Signature · 1 line
2.12.0
PATCH v4 3/6 bus: introduce device level DMA memory mapping
In reply to Shahaf ShulerThe DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the rte_malloc header).
The user should create a memory, register it through rte_extmem_register API, and call DMA map function in order to register such memory to the different devices.
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).
The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* call rte_extmem_register on the memory chunk.
* take a device, and query its rte_device.
* call the device specific mapping function for this device.
Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap APIs, leaving the rte device APIs as the preferred option for the user.
--- drivers/bus/pci/pci_common.c | 48 ++++++++++++++++++++++++++++ drivers/bus/pci/rte_bus_pci.h | 40 +++++++++++++++++++++++ lib/librte_eal/common/eal_common_dev.c | 34 ++++++++++++++++++++ lib/librte_eal/common/include/rte_bus.h | 44 +++++++++++++++++++++++++ lib/librte_eal/common/include/rte_dev.h | 47 +++++++++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 2 ++ 6 files changed, 215 insertions(+)
drivers/bus/pci/pci_common.c +48 -0
@@ -528,6 +528,52 @@ pci_unplug(struct rte_device *dev)return ret; } +static int +pci_dma_map (struct rte_device *dev,void *addr,uint64_t iova,size_t len) +{ +struct rte_pci_device *pdev =RTE_DEV_TO_PCI (dev); + +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ; +return -1 ; + } +if (pdev->driver->dma_map) +return pdev->driver->dma_map (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_map + (RTE_VFIO_DEFAULT_CONTAINER_FD , (uintptr_t )addr, + iova, len); + rte_errno =ENOTSUP ; +return -1 ; +} + +static int +pci_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova,size_t len) +{ +struct rte_pci_device *pdev =RTE_DEV_TO_PCI (dev); + +if (!pdev || !pdev->driver) { + rte_errno =EINVAL ; +return -1 ; + } +if (pdev->driver->dma_unmap) +return pdev->driver->dma_unmap (pdev, addr, iova, len); +/** +* In case driver don't provides any specific mapping +* try fallback to VFIO. +*/ +if (pdev->kdrv ==RTE_KDRV_VFIO ) +return rte_vfio_container_dma_unmap + (RTE_VFIO_DEFAULT_CONTAINER_FD , (uintptr_t )addr, + iova, len); + rte_errno =ENOTSUP ; +return -1 ; +} +struct rte_pci_bus rte_pci_bus = { .bus = { .scan = rte_pci_scan,
@@ -536,6 +582,8 @@ struct rte_pci_bus rte_pci_bus = { .plug = pci_plug, .unplug = pci_unplug, .parse = pci_parse, + .dma_map = pci_dma_map, + .dma_unmap = pci_dma_unmap, .get_iommu_class = rte_pci_get_iommu_class, .dev_iterate = rte_pci_dev_iterate, .hot_unplug_handler = pci_hot_unplug_handler,
drivers/bus/pci/rte_bus_pci.h +40 -0
@@ -114,6 +114,44 @@ typedef int (pci_probe_t)(struct rte_pci_driver *, struct rte_pci_device *);typedef int (pci_remove_t )(struct rte_pci_device *);/** +* Driver-specific DMA mapping. After a successful call the device +* will be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be mapped. +* @param iova +* Starting IOVA address of memory to be mapped. +* @param len +* Length of memory segment being mapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/ +typedef int (pci_dma_map_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** +* Driver-specific DMA un-mapping. After a successful call the device +* will not be able to read/write from/to this segment. +* +* @param dev +* Pointer to the PCI device. +* @param addr +* Starting virtual address of memory to be unmapped. +* @param iova +* Starting IOVA address of memory to be unmapped. +* @param len +* Length of memory segment being unmapped. +* @return +* - 0 On success. +* - Negative value and rte_errno is set otherwise. +*/ +typedef int (pci_dma_unmap_t )(struct rte_pci_device *dev,void *addr, +uint64_t iova,size_t len); + +/** * A structure describing a PCI driver. */ struct rte_pci_driver {
@@ -122,6 +160,8 @@ struct rte_pci_driver {struct rte_pci_bus *bus;/**< PCI bus reference. */ pci_probe_t *probe;/**< Device Probe function. */ pci_remove_t *remove;/**< Device Remove function. */ +pci_dma_map_t *dma_map;/**< device dma map function. */ +pci_dma_unmap_t *dma_unmap;/**< device dma unmap function. */ const struct rte_pci_id *id_table;/**< ID table, NULL terminated. */ uint32_t drv_flags;/**< Flags RTE_PCI_DRV_*. */ };
lib/librte_eal/common/eal_common_dev.c +34 -0
@@ -756,3 +756,37 @@ rte_dev_iterator_next(struct rte_dev_iterator *it)free (cls_str);return it->device; } + +int +rte_dev_dma_map (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->dma_map ==NULL || len ==0 ) { + rte_errno =ENOTSUP ; +return -1 ; + } +/* Memory must be registered through rte_extmem_* APIs */ +if (rte_mem_virt2memseg_list (addr) ==NULL ) { + rte_errno =EINVAL ; +return -1 ; + } + +return dev->bus->dma_map (dev, addr, iova, len); +} + +int +rte_dev_dma_unmap (struct rte_device *dev,void *addr,uint64_t iova, +size_t len) +{ +if (dev->bus->dma_unmap ==NULL || len ==0 ) { + rte_errno =ENOTSUP ; +return -1 ; + } +/* Memory must be registered through rte_extmem_* APIs */ +if (rte_mem_virt2memseg_list (addr) ==NULL ) { + rte_errno =EINVAL ; +return -1 ; + } + +return dev->bus->dma_unmap (dev, addr, iova, len); +}
lib/librte_eal/common/include/rte_bus.h +44 -0
56 lines not on this page · raw message
lib/librte_eal/common/include/rte_dev.h +47 -0
51 lines not on this page · raw message
lib/librte_eal/rte_eal_version.map +2 -0
@@ -277,6 +277,8 @@ EXPERIMENTAL { rte_class_unregister; rte_ctrl_thread_create; rte_delay_us_sleep; + rte_dev_dma_map; + rte_dev_dma_unmap; rte_dev_event_callback_process; rte_dev_event_callback_register; rte_dev_event_callback_unregister;
Signature · 1 line
2.12.0
Re: [PATCH v4 3/6] bus: introduce device level DMA memory mapping
Quoted 39 lines · Shahaf Shuler
The DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the
rte_mallocheader).
The user should create a memory, register it throughrte_extmem_registerAPI, and call DMA map function in order to register such memory to the different devices.The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* callrte_extmem_registeron the memory chunk.
* take a device, and query itsrte_device.
* call the device specific mapping function for this device.
Quoted 5 lines
Future work will deprecate the
rte_vfio_dma_mapandrte_vfio_dma_unmapAPIs, leaving the rte device APIs as the preferred option for the user.Signed-off-by Shahaf Shuler <[email protected]>---
Signature · 2 lines
Thanks, Anatoly
Re: [PATCH v4 3/6] bus: introduce device level DMA memory mapping
In reply to Shahaf Shuler10/03/2019 09:28, Shahaf Shuler:
Quoted 9 lines
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* callrte_extmem_registeron the memory chunk.
* take a device, and query itsrte_device.
* call the device specific mapping function for this device.
Should we make it documented somewhere?
Quoted 6 lines
+/**
+ * Device level DMA map function.
+ * After a successful call, the memory segment will be mapped to the
+ * given device.
+ *
+ * @note: Memory must be registered in advance using rte_extmem_* APIs.
Could we make more explicit that this function is part of the "external memory API"?
Re: [PATCH v4 3/6] bus: introduce device level DMA memory mapping
Quoted 17 lines
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Wednesday, March 13, 2019 11:56 AM
To: Shahaf Shuler <[email protected]>; [email protected]
Cc: [email protected]; Yongseok Koh <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH v4 3/6] bus: introduce device level DMA memory mapping10/03/2019 09:28, Shahaf Shuler:
Quoted 3 lines
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through
VFIO APIs.
Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Application usage with those APIs is quite simple:
* allocate memory
* callrte_extmem_registeron the memory chunk.
* take a device, and query itsrte_device.
* call the device specific mapping function for this device.Should we make it documented somewhere?
The full flow to work w/ external memory is documented at doc/guides/prog_guide/env_abstraction_layer.rst , Subchapter "Support for Externally Allocated Memory. The last commit in series update the right API to use.
Quoted 11 lines
Quoted 7 lines
+/**
+ * Device level DMA map function.
+ * After a successful call, the memory segment will be mapped to the
+ * given device.
+ *
+ * @note: Memory must be registered in advance using rte_extmem_*APIs.
Could we make more explicit that this function is part of the "external memory API"?
How do you suggest?
This function belongs to rte_dev therefore the rte_dev prefix. better rte_dev_extmem_dma_map ?
Quoted 2 lines
Re: [PATCH v4 3/6] bus: introduce device level DMA memory mapping
13/03/2019 12:12, Shahaf Shuler:
Quoted 7 lines
From: Thomas Monjalon <[email protected]>
Quoted 6 lines
Quoted 6 lines
+/**
+ * Device level DMA map function.
+ * After a successful call, the memory segment will be mapped to the
+ * given device.
+ *
+ * @note: Memory must be registered in advance using rte_extmem_*
Quoted 6 lines
Quoted 4 lines
APIs.
Could we make more explicit that this function is part of the "external memory API"?
How do you suggest?
There could be an explicit comment.
Quoted 1 line
This function belongs to
rte_devtherefore therte_devprefix. betterrte_dev_extmem_dma_map?
Not sure about the prefix. Anatoly?
Re: [PATCH v4 3/6] bus: introduce device level DMA memory mapping
Quoted 13 lines · Thomas Monjalon
13/03/2019 12:12, Shahaf Shuler:
Quoted 12 lines
From: Thomas Monjalon <[email protected]>
Quoted 10 lines
Quoted 6 lines
+/**
+ * Device level DMA map function.
+ * After a successful call, the memory segment will be mapped to the
+ * given device.
+ *
+ * @note: Memory must be registered in advance using rte_extmem_*APIs.
Could we make more explicit that this function is part of the "external memory API"?
Quoted 8 lines
Quoted 1 line
How do you suggest?
There could be an explicit comment.
Quoted 1 line
This function belongs to
rte_devtherefore therte_devprefix. betterrte_dev_extmem_dma_map?Not sure about the prefix. Anatoly?
IMO this is a dev API. The fact that its purpose is to use it with extmem is coincidental.
Signature · 2 lines
Thanks, Anatoly
Re: [PATCH v4 3/6] bus: introduce device level DMA memory mapping
In reply to Shahaf Shuler13/03/2019 12:12, Shahaf Shuler:
Quoted 13 lines
From: Thomas Monjalon <[email protected]>
Quoted 12 lines
10/03/2019 09:28, Shahaf Shuler:
Quoted 3 lines
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through
VFIO APIs.
Quoted 6 lines
Application usage with those APIs is quite simple:
* allocate memory
* callrte_extmem_registeron the memory chunk.
* take a device, and query itsrte_device.
* call the device specific mapping function for this device.
Quoted 4 lines
Quoted 1 line
Should we make it documented somewhere?
The full flow to work w/ external memory is documented at
doc/guides/prog_guide/env_abstraction_layer.rst, Subchapter "Support for Externally Allocated Memory. The last commit in series update the right API to use.
OK, then I will move this doc update in this patch.
PATCH v4 4/6 net/mlx5: refactor external memory registration
In reply to Shahaf ShulerMove the memory region creation to a separate function to prepare the ground for the reuse of it on the PCI driver map and unmap functions.
--- drivers/net/mlx5/mlx5_mr.c | 86 +++++++++++++++++++++++++++-------------- 1 file changed, 57 insertions(+), 29 deletions(-)
drivers/net/mlx5/mlx5_mr.c +57 -29
@@ -1109,6 +1109,58 @@ mlx5_mr_flush_local_cache(struct mlx5_mr_ctrl *mr_ctrl) }/** +* Creates a memory region for external memory, that is memory which is not +* part of the DPDK memory segments. +* +* @param dev +* Pointer to the ethernet device. +* @param addr +* Starting virtual address of memory. +* @param len +* Length of memory segment being mapped. +* @param socked_id +* Socket to allocate heap memory for the control structures. +* +* @return +* Pointer to MR structure on success, NULL otherwise. +*/ +static struct mlx5_mr * +mlx5_create_mr_ext (struct rte_eth_dev *dev,uintptr_t addr,size_t len, +int socket_id) +{ +struct mlx5_priv *priv = dev->data->dev_private; +struct mlx5_mr *mr =NULL ; + + mr =rte_zmalloc_socket (NULL , +RTE_ALIGN_CEIL (sizeof (*mr), +RTE_CACHE_LINE_SIZE ), +RTE_CACHE_LINE_SIZE , socket_id); +if (mr ==NULL ) +return NULL ; + mr->ibv_mr = mlx5_glue->reg_mr (priv->pd, (void *)addr, len, +IBV_ACCESS_LOCAL_WRITE ); +if (mr->ibv_mr ==NULL ) { +DRV_LOG (WARNING , +"port %u fail to create a verbs MR for address (%p)" , + dev->data->port_id, (void *)addr); +rte_free (mr); +return NULL ; + } + mr->msl =NULL ;/* Mark it is external memory. */ + mr->ms_bmp =NULL ; + mr->ms_n =1 ; + mr->ms_bmp_n =1 ; +DRV_LOG (DEBUG , +"port %u MR CREATED (%p) for external memory %p:\n" +" [0x%" PRIxPTR", 0x%" PRIxPTR")," +" lkey=0x%x base_idx=%u ms_n=%u, ms_bmp_n=%u" , + dev->data->port_id, (void *)mr, (void *)addr, + addr, addr + len,rte_cpu_to_be_32 (mr->ibv_mr->lkey), + mr->ms_base_idx, mr->ms_n, mr->ms_bmp_n); +return mr; +} + +/** * Called during rte_mempool_mem_iter() by mlx5_mr_update_ext_mp(). * * Externally allocated chunk is registered and a MR is created for the chunk.
@@ -1142,43 +1194,19 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque,rte_rwlock_read_unlock (&priv->mr.rwlock);if (lkey !=UINT32_MAX )return ; - mr =rte_zmalloc_socket (NULL , -RTE_ALIGN_CEIL (sizeof (*mr), -RTE_CACHE_LINE_SIZE ), -RTE_CACHE_LINE_SIZE , mp->socket_id); -if (mr ==NULL ) { -DRV_LOG (WARNING , -"port %u unable to allocate memory for a new MR of" -" mempool (%s)." , - dev->data->port_id, mp->name); - data->ret =-1 ; -return ; - }DRV_LOG (DEBUG ,"port %u register MR for chunk #%d of mempool (%s)" , dev->data->port_id, mem_idx, mp->name); - mr->ibv_mr = mlx5_glue->reg_mr (priv->pd, (void *)addr, len, -IBV_ACCESS_LOCAL_WRITE ); -if (mr->ibv_mr ==NULL ) { + mr =mlx5_create_mr_ext (dev, addr, len, mp->socket_id); +if (!mr) {DRV_LOG (WARNING , -"port %u fail to create a verbs MR for address (%p)" , - dev->data->port_id, (void *)addr); -rte_free (mr); +"port %u unable to allocate a new MR of" +" mempool (%s)." , + dev->data->port_id, mp->name); data->ret =-1 ;return ; } - mr->msl =NULL ;/* Mark it is external memory. */ - mr->ms_bmp =NULL ; - mr->ms_n =1 ; - mr->ms_bmp_n =1 ;rte_rwlock_write_lock (&priv->mr.rwlock);LIST_INSERT_HEAD (&priv->mr.mr_list, mr, mr); -DRV_LOG (DEBUG , -"port %u MR CREATED (%p) for external memory %p:\n" -" [0x%" PRIxPTR", 0x%" PRIxPTR")," -" lkey=0x%x base_idx=%u ms_n=%u, ms_bmp_n=%u" , - dev->data->port_id, (void *)mr, (void *)addr, - addr, addr + len,rte_cpu_to_be_32 (mr->ibv_mr->lkey), - mr->ms_base_idx, mr->ms_n, mr->ms_bmp_n);/* Insert to the global cache table. */ mr_insert_dev_cache (dev, mr);rte_rwlock_write_unlock (&priv->mr.rwlock);
Signature · 1 line
2.12.0
PATCH v4 5/6 net/mlx5: support PCI device DMA map and unmap
In reply to Shahaf ShulerThe implementation reuses the external memory registration work done by commit[1].
Note about representors:
The current representor design will not work
with those map and unmap functions. The reason is that for representors we have multiple IB devices share the same PCI function, so mapping will happen only on one of the representors and not all of them.
While it is possible to implement such support, the IB representor design is going to be changed during DPDK19.05. The new design will have a single IB device for all representors, hence sharing of a single memory region between all representors will be possible.
[1]
commit 7e43a32ee060
("net/mlx5: support externally allocated static memory")
--- drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 139 ++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 5 ++ 3 files changed, 146 insertions(+)
drivers/net/mlx5/mlx5.c +2 -0
@@ -1632,6 +1632,8 @@ static struct rte_pci_driver mlx5_driver = { .id_table = mlx5_pci_id_map, .probe = mlx5_pci_probe, .remove = mlx5_pci_remove, + .dma_map = mlx5_dma_map, + .dma_unmap = mlx5_dma_unmap, .drv_flags = (RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV | RTE_PCI_DRV_PROBE_AGAIN), };
drivers/net/mlx5/mlx5_mr.c +139 -0
151 lines not on this page · raw message
drivers/net/mlx5/mlx5_rxtx.h +5 -0
@@ -28,6 +28,7 @@#include <rte_atomic.h> #include <rte_spinlock.h> #include <rte_io.h> +#include <rte_bus_pci.h> #include "mlx5_utils.h" #include "mlx5.h"
@@ -367,6 +368,10 @@ uint32_t mlx5_rx_addr2mr_bh(struct mlx5_rxq_data *rxq, uintptr_t addr);uint32_t mlx5_tx_mb2mr_bh (struct mlx5_txq_data *txq,struct rte_mbuf *mb);uint32_t mlx5_tx_update_ext_mp (struct mlx5_txq_data *txq,uintptr_t addr,struct rte_mempool *mp); +int mlx5_dma_map (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len); +int mlx5_dma_unmap (struct rte_pci_device *pdev,void *addr,uint64_t iova, +size_t len);/** * Provide safe 64bit store operation to mlx5 UAR region for both 32bit and
Signature · 1 line
2.12.0
PATCH v4 6/6 doc: deprecation notice for VFIO DMA map APIs
In reply to Shahaf ShulerAs those should be replaced by rte_dev_dma_map and rte_dev_dma_unmap APIs.
--- doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-)
doc/guides/prog_guide/env_abstraction_layer.rst +1 -1
@@ -282,7 +282,7 @@ The expected workflow is as follows: - If IOVA table is not specified, IOVA addresses will be assumed to be unavailable - Other processes must attach to the memory area before they can use it -* Perform DMA mapping with ``rte_vfio_dma_map`` if needed +* Perform DMA mapping with ``rte_dev_dma_map`` if needed * Use the memory area in your application * If memory area is no longer needed, it can be unregistered - If the area was mapped for DMA, unmapping must be performed before
doc/guides/rel_notes/deprecation.rst +4 -0
@@ -35,6 +35,10 @@ Deprecation Notices + ``rte_eal_devargs_type_count`` +* vfio: removal of ``rte_vfio_dma_map`` and ``rte_vfio_dma_unmap`` APIs which + have been replaced with ``rte_dev_dma_map`` and ``rte_dev_dma_unmap`` + functions. The due date for the removal targets DPDK 20.02. + * pci: Several exposed functions are misnamed. The following functions are deprecated starting from v17.11 and are replaced:
Signature · 1 line
2.12.0
Re: [PATCH v4 6/6] doc: deprecation notice for VFIO DMA map APIs
Quoted 5 lines · Shahaf Shuler
As those should be replaced by
rte_dev_dma_mapandrte_dev_dma_unmapAPIs.Signed-off-by Shahaf Shuler <[email protected]>---
Signature · 2 lines
Thanks, Anatoly
Rami Rosen
Acked-byRe: [PATCH v4 6/6] doc: deprecation notice for VFIO DMA map APIs
Quoted 7 lines · Shahaf Shuler
Quoted 5 lines
As those should be replaced by
rte_dev_dma_mapandrte_dev_dma_unmapAPIs.Signed-off-by Shahaf Shuler <[email protected]>--
Re: [dpdk-dev] [PATCH v4 6/6] doc: deprecation notice for VFIO DMA map APIs
In reply to Shahaf ShulerHello Shahaf,
Quoted 28 lines · Shahaf Shuler
As those should be replaced by
rte_dev_dma_mapandrte_dev_dma_unmapAPIs.Signed-off-by Shahaf Shuler <[email protected]>--- doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-)doc/guides/prog_guide/env_abstraction_layer.rst +1
-1@@ -282,7 +282,7 @@ The expected workflow is as follows: - If IOVA table is not specified, IOVA addresses will be assumed to be unavailable - Other processes must attach to the memory area before they can use it -* Perform DMA mapping with ``rte_vfio_dma_map`` if needed +* Perform DMA mapping with ``rte_dev_dma_map`` if needed * Use the memory area in your application * If memory area is no longer needed, it can be unregistered - If the area was mapped for DMA, unmapping must be performed beforedoc/guides/rel_notes/deprecation.rst +0
-0
doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 44
Quoted 10 lines
doc/guides/rel_notes/deprecation.rst +4
-0@@ -35,6 +35,10 @@ Deprecation Notices + ``rte_eal_devargs_type_count`` +* vfio: removal of ``rte_vfio_dma_map`` and ``rte_vfio_dma_unmap`` APIs which + have been replaced with ``rte_dev_dma_map`` and ``rte_dev_dma_unmap`` + functions. The due date for the removal targets DPDK 20.02. + * pci: Several exposed functions are misnamed. The following functions are deprecated starting from v17.11 and are replaced:
With the ABI freeze that is going to happen in 19.11, this can't happen in 20.02.
What would work best from your pov?
I can't see any in-tree user of rte_vfio_dma_*map, do you know of users of this api?
Thanks.
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v4 6/6] doc: deprecation notice for VFIO DMA map APIs
Hi David,
Tuesday, October 1, 2019 6:20 PM, David Marchand:
Quoted 53 lines
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH v4 6/6] doc: deprecation notice for VFIO DMA map APIsHello Shahaf,
Quoted 1 line · Shahaf Shuler
As those should be replaced by
rte_dev_dma_mapand
rte_dev_dma_unmapQuoted 30 lines
APIs.
Signed-off-by Shahaf Shuler <[email protected]>--- doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-)doc/guides/prog_guide/env_abstraction_layer.rst +0
-0@@ -282,7 +282,7 @@ The expected workflow is as follows: - If IOVA table is not specified, IOVA addresses will be assumed to be unavailable - Other processes must attach to the memory area before they canuse it
doc/guides/prog_guide/env_abstraction_layer.rst +1
-1-* Perform DMA mapping with ``rte_vfio_dma_map`` if needed +* Perform DMA mapping with ``rte_dev_dma_map`` if needed * Use the memory area in your application * If memory area is no longer needed, it can be unregistered - If the area was mapped for DMA, unmapping must be performedbefore diff --git
a/doc/guides/rel_notes/deprecation.rstb/doc/guides/rel_notes/deprecation.rst
index1b4fcb7e64..48ec4fee88100644doc/guides/rel_notes/deprecation.rst +0
-0@@ -35,6 +35,10 @@ Deprecation Noticesdoc/guides/rel_notes/deprecation.rst Deprecation Notices
Quoted 10 lines
doc/guides/rel_notes/deprecation.rst +6
-0+ ``rte_eal_devargs_type_count`` +* vfio: removal of ``rte_vfio_dma_map`` and ``rte_vfio_dma_unmap`` +APIs which + have been replaced with ``rte_dev_dma_map`` and +``rte_dev_dma_unmap`` + functions. The due date for the removal targets DPDK 20.02. + * pci: Several exposed functions are misnamed. The following functions are deprecated starting from v17.11 and arereplaced:
Quoted 2 lines
Quoted 4 lines
With the ABI freeze that is going to happen in 19.11, this can't happen in 20.02.
What would work best from your pov?
I have no object (even prefer) to remove them at 19.11.
At the time I sent the deprecation I was requested to provide more time for application to adopt.
Quoted 3 lines
I can't see any in-tree user of rte_vfio_dma_*map, do you know of users of this api?
There is one - VPP. They don't use DPDK memory subsystem at all, rather use they own allocated memory and map all, wrongly, w/ above APIs.
If all agree - we can remove those now.
Quoted 6 lines
Thanks.
--
David Marchand
Re: [dpdk-dev] [PATCH v4 6/6] doc: deprecation notice for VFIO DMA map APIs
Quoted 62 lines · Shahaf Shuler
Hi David,
Tuesday, October 1, 2019 6:20 PM, David Marchand:
Quoted 53 lines
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH v4 6/6] doc: deprecation notice for VFIO DMA map APIsHello Shahaf,
Quoted 1 line · Shahaf Shuler
As those should be replaced by
rte_dev_dma_mapand
rte_dev_dma_unmapQuoted 30 lines
APIs.
Signed-off-by Shahaf Shuler <[email protected]>--- doc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-)doc/guides/prog_guide/env_abstraction_layer.rst +0
-0@@ -282,7 +282,7 @@ The expected workflow is as follows: - If IOVA table is not specified, IOVA addresses will be assumed to be unavailable - Other processes must attach to the memory area before they canuse it
doc/guides/prog_guide/env_abstraction_layer.rst +1
-1-* Perform DMA mapping with ``rte_vfio_dma_map`` if needed +* Perform DMA mapping with ``rte_dev_dma_map`` if needed * Use the memory area in your application * If memory area is no longer needed, it can be unregistered - If the area was mapped for DMA, unmapping must be performedbefore diff --git
a/doc/guides/rel_notes/deprecation.rstb/doc/guides/rel_notes/deprecation.rst
index1b4fcb7e64..48ec4fee88100644doc/guides/rel_notes/deprecation.rst +0
-0@@ -35,6 +35,10 @@ Deprecation Noticesdoc/guides/rel_notes/deprecation.rst Deprecation Notices
Quoted 10 lines
doc/guides/rel_notes/deprecation.rst +6
-0+ ``rte_eal_devargs_type_count`` +* vfio: removal of ``rte_vfio_dma_map`` and ``rte_vfio_dma_unmap`` +APIs which + have been replaced with ``rte_dev_dma_map`` and +``rte_dev_dma_unmap`` + functions. The due date for the removal targets DPDK 20.02. + * pci: Several exposed functions are misnamed. The following functions are deprecated starting from v17.11 and arereplaced:
Quoted 2 lines
Quoted 4 lines
With the ABI freeze that is going to happen in 19.11, this can't happen in 20.02.
What would work best from your pov?
Quoted 8 lines
I have no object (even prefer) to remove them at 19.11.
At the time I sent the deprecation I was requested to provide more time for application to adopt.Quoted 3 lines
I can't see any in-tree user of rte_vfio_dma_*map, do you know of users of this api?
There is one - VPP. They don't use DPDK memory subsystem at all, rather use they own allocated memory and map all, wrongly, w/ above APIs.
Thanks Shahaf.
I cannot see VPP involved people copied in this thread. It would have been great to involve them at the time.
Ray, can you reply on this topic (replacement of rte_vfio_dma_map) ? Or could you serve as a gateway/copy the vpp guys?
Thanks.
--
David Marchand
Hello Shahaf,
Thanks for taking my remarks into account. You can add my acked-by if you need to the series (not really for mlx5 PMD but you get the idea).
BR,
Quoted 97 lines · Shahaf Shuler
The DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the
rte_mallocheader).
The user should create a memory, register it throughrte_extmem_registerAPI, and call DMA map function in order to register such memory to the different devices.The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* callrte_extmem_registeron the memory chunk.
* take a device, and query itsrte_device.
* call the device specific mapping function for this device.Future work will deprecate the
rte_vfio_dma_mapandrte_vfio_dma_unmapAPIs, leaving the rte device APIs as the preferred option for the user.On v4:
- Changedrte_dev_dma_maperrno to ENOTSUP in case bus doesn't support DMA map API.On v3:
- Fixed compilation issue on freebsd.
- Fixed forgottenrte_bus_dma_maptorte_dev_dma_map.
- Removed__rte_deprecatedfrom vfio function till the time therte_dev_dma_mapwill be non-experimental.
- Changed error return value to always be -1, with proper errno.
- Usedrte_mem_virt2memseg_listinstead ofrte_mem_virt2memsegto verify memory is registered.
- Added above check also ondma_unmapcalls.
- Added note in the API the memory must be registered in advance.
- Added debug log to report the case memory mapping to vfio was skipped.On v2:
- Added warn in release notes about the API change in vfio.
- Moved function doc to prototype declaration.
- Useddma_mapanddma_unmapinstead of map and unmap.
- UsedRTE_VFIO_DEFAULT_CONTAINER_FDinstead of -1 fixed value.
- Moved bus function toeal_common_dev.cand renamed them properly.
- Changed eth device iterator to useRTE_DEV_FOREACH.
- Enforced memory is registered with rte_extmem_* prior to mapping.
- Used EEXIST as the only possible return value from type1 vfio IOMMU mapping.[1] https://patches.dpdk.org/patch/47796/
Shahaf Shuler (6):
vfio: allow DMA map of memory for the default vfio fd
vfio: don't fail to DMA map if memory is already mapped
bus: introduce device level DMA memory mapping net/mlx5: refactor external memory registration net/mlx5: support PCI device DMA map and unmap
doc: deprecation notice for VFIO DMA map APIsdoc/guides/prog_guide/env_abstraction_layer.rst | 2 +- doc/guides/rel_notes/deprecation.rst | 4 + doc/guides/rel_notes/release_19_05.rst | 3 + drivers/bus/pci/pci_common.c | 48 ++++ drivers/bus/pci/rte_bus_pci.h | 40 ++++ drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 225 ++++++++++++++++--- drivers/net/mlx5/mlx5_rxtx.h | 5 + lib/librte_eal/common/eal_common_dev.c | 34 +++ lib/librte_eal/common/include/rte_bus.h | 44 ++++ lib/librte_eal/common/include/rte_dev.h | 47 ++++ lib/librte_eal/common/include/rte_vfio.h | 8 +- lib/librte_eal/linuxapp/eal/eal_vfio.c | 42 +++- lib/librte_eal/rte_eal_version.map | 2 + 14 files changed, 468 insertions(+), 38 deletions(-)Signature · 1 line
2.12.0
Signature · 2 lines
Gaëtan Rivet 6WIND
Thomas Monjalon
Applied10/03/2019 09:27, Shahaf Shuler:
Quoted 42 lines
The DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages). This memory is allocated by the DPDK libraries, included in the DPDK memory system (memseg lists) and automatically DMA mapped by the DPDK layers.
2. Use memory allocated by the user and register to the DPDK memory systems. Upon registration of memory, the DPDK layers will DMA map it to all needed devices. After registration, allocation of this memory will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory system. This is for users who wants to have tight control on this memory (e.g. avoid the
rte_mallocheader).
The user should create a memory, register it throughrte_extmem_registerAPI, and call DMA map function in order to register such memory to the different devices.The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO (
rte_vfio_dma_map). While VFIO is common, there are other vendors which use different ways to map memory (e.g. Mellanox and NXP).The work in this patch moves the DMA mapping to vendor agnostic APIs. Device level DMA map and unmap APIs were added. Implementation of those APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap functions to be used for the mapping. In case the driver doesn't provide any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* callrte_extmem_registeron the memory chunk.
* take a device, and query itsrte_device.
* call the device specific mapping function for this device.Future work will deprecate the
rte_vfio_dma_mapandrte_vfio_dma_unmapAPIs, leaving the rte device APIs as the preferred option for the user.
Quoted 7 lines
Shahaf Shuler (6):
vfio: allow DMA map of memory for the default vfio fd
vfio: don't fail to DMA map if memory is already mapped
bus: introduce device level DMA memory mapping net/mlx5: refactor external memory registration net/mlx5: support PCI device DMA map and unmap
doc: deprecation notice for VFIO DMA map APIs
Applied, thanks