Introduce Microsoft Azure Network Adapter (MANA) RDMA driver
Subsystems (5)
PATCH v9 1/12 net: mana: Add support for auxiliary device
In reply to [email protected]From: Long Li <[email protected]>
In preparation for supporting MANA RDMA driver, add support for auxiliary device in the Ethernet driver. The RDMA device is modeled as an auxiliary device to the Ethernet device.
---
Change log:
v3: define mana_adev_idx_alloc and mana_adev_idx_free as static
v7: fix a bug that may assign a negative value to adev->id
drivers/net/ethernet/microsoft/Kconfig | 1 + drivers/net/ethernet/microsoft/mana/gdma.h | 2 + .../ethernet/microsoft/mana/mana_auxiliary.h | 10 +++ drivers/net/ethernet/microsoft/mana/mana_en.c | 83 ++++++++++++++++++- 4 files changed, 95 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/microsoft/mana/mana_auxiliary.h
drivers/net/ethernet/microsoft/Kconfig +1 -0
@@ -19,6 +19,7 @@ config MICROSOFT_MANA tristate "Microsoft Azure Network Adapter (MANA) support" depends on PCI_MSI && X86_64 depends on PCI_HYPERV + select AUXILIARY_BUS help This driver supports Microsoft Azure Network Adapter (MANA). So far, the driver is only supported on X86_64.
drivers/net/ethernet/microsoft/mana/gdma.h +2 -0
@@ -204,6 +204,8 @@ struct gdma_dev {/* GDMA driver specific pointer */ void *driver_data; + +struct auxiliary_device *adev; };#define MINIMUM_SUPPORTED_PAGE_SIZE PAGE_SIZE
drivers/net/ethernet/microsoft/mana/mana_auxiliary.h new file +10 -0
@@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (c) 2022, Microsoft Corporation. */ + +#include "mana.h" +#include <linux/auxiliary_bus.h> + +struct mana_adev { +struct auxiliary_device adev; +struct gdma_dev *mdev; +};
drivers/net/ethernet/microsoft/mana/mana_en.c +82 -1
113 lines not on this page · raw message
Signature · 1 line
2.17.1
v9 cover Introduce Microsoft Azure Network Adapter (MANA) RDMA driver
From: Long Li <[email protected]>
This patchset implements a RDMA driver for Microsoft Azure Network Adapter (MANA). In MANA, the RDMA device is modeled as an auxiliary device to the Ethernet device.
The first 11 patches modify the MANA Ethernet driver to support RDMA driver. The last patch implementes the RDMA driver.
The user-mode of the driver is being reviewed at:
https://github.com/linux-rdma/rdma-core/pull/1177
Ajay Sharma (3):
net: mana: Set the DMA device max segment size
net: mana: Define and process GDMA response code
GDMA_STATUS_MORE_ENTRIES
net: mana: Define data structures for protection domain and memory
registration
Long Li (9):
net: mana: Add support for auxiliary device
net: mana: Record the physical address for doorbell page region
net: mana: Handle vport sharing between devices
net: mana: Export Work Queue functions for use by RDMA driver
net: mana: Record port number in netdev
net: mana: Move header files to a common location
net: mana: Define max values for SGL entries
net: mana: Define data structures for allocating doorbell page from
GDMA
RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter
MAINTAINERS | 10 +
drivers/infiniband/Kconfig | 1 +
drivers/infiniband/hw/Makefile | 1 +
drivers/infiniband/hw/mana/Kconfig | 10 +
drivers/infiniband/hw/mana/Makefile | 4 +
drivers/infiniband/hw/mana/cq.c | 79 +++
drivers/infiniband/hw/mana/device.c | 117 ++++
drivers/infiniband/hw/mana/main.c | 507 ++++++++++++++++++
drivers/infiniband/hw/mana/mana_ib.h | 162 ++++++
drivers/infiniband/hw/mana/mr.c | 197 +++++++
drivers/infiniband/hw/mana/qp.c | 506 +++++++++++++++++
drivers/infiniband/hw/mana/wq.c | 115 ++++
drivers/net/ethernet/microsoft/Kconfig | 1 +
.../net/ethernet/microsoft/mana/gdma_main.c | 40 +-
.../net/ethernet/microsoft/mana/hw_channel.c | 6 +-
.../net/ethernet/microsoft/mana/mana_bpf.c | 2 +-
drivers/net/ethernet/microsoft/mana/mana_en.c | 175 +++++-
.../ethernet/microsoft/mana/mana_ethtool.c | 2 +-
.../net/ethernet/microsoft/mana/shm_channel.c | 2 +-
.../microsoft => include/net}/mana/gdma.h | 158 +++++-
.../net}/mana/hw_channel.h | 0
.../microsoft => include/net}/mana/mana.h | 23 +-
include/net/mana/mana_auxiliary.h | 10 +
.../net}/mana/shm_channel.h | 0
include/uapi/rdma/ib_user_ioctl_verbs.h | 1 +
include/uapi/rdma/mana-abi.h | 66 +++
26 files changed, 2150 insertions(+), 45 deletions(-)
create mode 100644 drivers/infiniband/hw/mana/Kconfig
create mode 100644 drivers/infiniband/hw/mana/Makefile
create mode 100644 drivers/infiniband/hw/mana/cq.c
create mode 100644 drivers/infiniband/hw/mana/device.c
create mode 100644 drivers/infiniband/hw/mana/main.c
create mode 100644 drivers/infiniband/hw/mana/mana_ib.h
create mode 100644 drivers/infiniband/hw/mana/mr.c
create mode 100644 drivers/infiniband/hw/mana/qp.c
create mode 100644 drivers/infiniband/hw/mana/wq.c
rename {drivers/net/ethernet/microsoft => include/net}/mana/gdma.h (80%)
rename {drivers/net/ethernet/microsoft => include/net}/mana/hw_channel.h (100%)
rename {drivers/net/ethernet/microsoft => include/net}/mana/mana.h (95%)
create mode 100644 include/net/mana/mana_auxiliary.h
rename {drivers/net/ethernet/microsoft => include/net}/mana/shm_channel.h (100%)
create mode 100644 include/uapi/rdma/mana-abi.hSignature · 1 line
2.17.1
PATCH v9 3/12 net: mana: Handle vport sharing between devices
From: Long Li <[email protected]>
For outgoing packets, the PF requires the VF to configure the vport with corresponding protection domain and doorbell ID for the kernel or user context. The vport can't be shared between different contexts.
Implement the logic to exclusively take over the vport by either the Ethernet device or RDMA device.
---
Change log:
v2: use refcount instead of directly using atomic variables
v4: change to mutex to avoid possible race with refcount
v5: add detailed comments explaining vport sharing, use EXPORT_SYMBOL_NS
v6: rebased to rdma-next
drivers/net/ethernet/microsoft/mana/mana.h | 7 +++ drivers/net/ethernet/microsoft/mana/mana_en.c | 53 ++++++++++++++++++- 2 files changed, 58 insertions(+), 2 deletions(-)
drivers/net/ethernet/microsoft/mana/mana.h +7 -0
@@ -380,6 +380,10 @@ struct mana_port_context {mana_handle_t port_handle;mana_handle_t pf_filter_handle; +/* Mutex for sharing access to vport_use_count */ +struct mutex vport_mutex; +int vport_use_count; +u16 port_idx;bool port_is_up;
@@ -631,4 +635,7 @@ struct mana_tx_package { struct gdma_posted_wqe_info wqe_info; }; +int mana_cfg_vport (struct mana_port_context *apc,u32 protection_dom_id, +u32 doorbell_pg_id); +void mana_uncfg_vport (struct mana_port_context *apc);#endif /* _MANA_H */
drivers/net/ethernet/microsoft/mana/mana_en.c +51 -2
91 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v9 2/12 net: mana: Record the physical address for doorbell page region
In reply to [email protected]From: Long Li <[email protected]>
For supporting RDMA device with multiple user contexts with their individual doorbell pages, record the start address of doorbell page region for use by the RDMA driver to allocate user context doorbell IDs.
---
Change log:
v6: rebased to rdma-next
drivers/net/ethernet/microsoft/mana/gdma.h | 2 ++ drivers/net/ethernet/microsoft/mana/gdma_main.c | 4 ++++ 2 files changed, 6 insertions(+)
drivers/net/ethernet/microsoft/mana/gdma.h +2 -0
@@ -351,9 +351,11 @@ struct gdma_context {u32 test_event_eq_id;bool is_pf; +phys_addr_t bar0_pa;void __iomem *bar0_va;void __iomem *shm_base;void __iomem *db_page_base; +phys_addr_t phys_db_page_base;u32 db_page_size;/* Shared memory chanenl (used to bootstrap HWC) */
drivers/net/ethernet/microsoft/mana/gdma_main.c +4 -0
@@ -44,6 +44,9 @@ static void mana_gd_init_vf_regs(struct pci_dev *pdev) gc->db_page_base = gc->bar0_va +mana_gd_r64 (gc,GDMA_REG_DB_PAGE_OFFSET ); + gc->phys_db_page_base = gc->bar0_pa + +mana_gd_r64 (gc,GDMA_REG_DB_PAGE_OFFSET ); + gc->shm_base = gc->bar0_va +mana_gd_r64 (gc,GDMA_REG_SHM_OFFSET ); }
@@ -1367,6 +1370,7 @@ static int mana_gd_probe(struct pci_dev *pdev, const struct pci_device_id *ent)mutex_init (&gc->eq_test_event_mutex);pci_set_drvdata (pdev, gc); + gc->bar0_pa =pci_resource_start (pdev,0 ); bar0_va =pci_iomap (pdev, bar,0 );if (!bar0_va)
Signature · 1 line
2.17.1
PATCH v9 4/12 net: mana: Set the DMA device max segment size
In reply to [email protected]From: Ajay Sharma <[email protected]>
MANA hardware doesn't have any restrictions on the DMA segment size, set it to the max allowed value.
---
Change log:
v2: Use the max allowed value as the hardware doesn't have any limit
drivers/net/ethernet/microsoft/mana/gdma_main.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/net/ethernet/microsoft/mana/gdma_main.c +6 -0
@@ -1363,6 +1363,12 @@ static int mana_gd_probe(struct pci_dev *pdev, const struct pci_device_id *ent)if (err) goto release_region; + err =dma_set_max_seg_size (&pdev->dev,UINT_MAX ); +if (err) { +dev_err (&pdev->dev,"Failed to set dma device segment size\n" ); + goto release_region; + } + err = -ENOMEM ; gc =vzalloc (sizeof (*gc));if (!gc)
Signature · 1 line
2.17.1
PATCH v9 5/12 net: mana: Export Work Queue functions for use by RDMA driver
In reply to [email protected]From: Long Li <[email protected]>
RDMA device may need to create Ethernet device queues for use by Queue Pair type RAW. This allows a user-mode context accesses Ethernet hardware queues. Export the supporting functions for use by the RDMA driver.
---
Change log:
v3: format/coding style changes
v5: remove unused defintions, use EXPORT_SYMBOL_NS, rearrange some defintions to a later patch in the series
drivers/net/ethernet/microsoft/mana/gdma_main.c | 1 + drivers/net/ethernet/microsoft/mana/mana.h | 9 +++++++++ drivers/net/ethernet/microsoft/mana/mana_en.c | 16 +++++++++------- 3 files changed, 19 insertions(+), 7 deletions(-)
drivers/net/ethernet/microsoft/mana/gdma_main.c +1 -0
@@ -152,6 +152,7 @@ int mana_gd_send_request(struct gdma_context *gc, u32 req_len, const void *req,return mana_hwc_send_request (hwc, req_len, req, resp_len, resp); } +EXPORT_SYMBOL_NS (mana_gd_send_request,NET_MANA );int mana_gd_alloc_memory (struct gdma_context *gc,unsigned int length,struct gdma_mem_info *gmi)
drivers/net/ethernet/microsoft/mana/mana.h +9 -0
@@ -635,6 +635,15 @@ struct mana_tx_package {struct gdma_posted_wqe_info wqe_info; }; +int mana_create_wq_obj (struct mana_port_context *apc, +mana_handle_t vport, +u32 wq_type,struct mana_obj_spec *wq_spec, +struct mana_obj_spec *cq_spec, +mana_handle_t *wq_obj); + +void mana_destroy_wq_obj (struct mana_port_context *apc,u32 wq_type, +mana_handle_t wq_obj); +int mana_cfg_vport (struct mana_port_context *apc,u32 protection_dom_id,u32 doorbell_pg_id);void mana_uncfg_vport (struct mana_port_context *apc);
drivers/net/ethernet/microsoft/mana/mana_en.c +9 -7
@@ -792,11 +792,11 @@ static int mana_cfg_vport_steering(struct mana_port_context *apc,return err; } -static int mana_create_wq_obj (struct mana_port_context *apc, -mana_handle_t vport, -u32 wq_type,struct mana_obj_spec *wq_spec, -struct mana_obj_spec *cq_spec, -mana_handle_t *wq_obj) +int mana_create_wq_obj (struct mana_port_context *apc, +mana_handle_t vport, +u32 wq_type,struct mana_obj_spec *wq_spec, +struct mana_obj_spec *cq_spec, +mana_handle_t *wq_obj) {struct mana_create_wqobj_resp resp = {};struct mana_create_wqobj_req req = {};
@@ -845,9 +845,10 @@ static int mana_create_wq_obj(struct mana_port_context *apc, out:return err; } +EXPORT_SYMBOL_NS (mana_create_wq_obj,NET_MANA ); -static void mana_destroy_wq_obj (struct mana_port_context *apc,u32 wq_type, -mana_handle_t wq_obj) +void mana_destroy_wq_obj (struct mana_port_context *apc,u32 wq_type, +mana_handle_t wq_obj) {struct mana_destroy_wqobj_resp resp = {};struct mana_destroy_wqobj_req req = {};
@@ -872,6 +873,7 @@ static void mana_destroy_wq_obj(struct mana_port_context *apc, u32 wq_type,netdev_err (ndev,"Failed to destroy WQ object: %d, 0x%x\n" , err, resp.hdr.status); } +EXPORT_SYMBOL_NS (mana_destroy_wq_obj,NET_MANA );static void mana_destroy_eq (struct mana_context *ac) {
Signature · 1 line
2.17.1
PATCH v9 6/12 net: mana: Record port number in netdev
In reply to [email protected]From: Long Li <[email protected]>
The port number is useful for user-mode application to identify this net device based on port index. Set to the correct value in ndev.
--- drivers/net/ethernet/microsoft/mana/mana_en.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/ethernet/microsoft/mana/mana_en.c +1 -0
@@ -2133,6 +2133,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx, ndev->max_mtu = ndev->mtu; ndev->min_mtu = ndev->mtu; ndev->needed_headroom =MANA_HEADROOM ; + ndev->dev_port = port_idx;SET_NETDEV_DEV (ndev, gc->dev);netif_carrier_off (ndev);
Signature · 1 line
2.17.1
PATCH v9 7/12 net: mana: Move header files to a common location
In reply to [email protected]From: Long Li <[email protected]>
In preparation to add MANA RDMA driver, move all the required header files to a common location for use by both Ethernet and RDMA drivers.
---
Change log:
v2: Move headers to include/net/mana, instead of include/linux/mana
MAINTAINERS | 1 +
drivers/net/ethernet/microsoft/mana/gdma_main.c | 2 +-
drivers/net/ethernet/microsoft/mana/hw_channel.c | 4 ++--
drivers/net/ethernet/microsoft/mana/mana_bpf.c | 2 +-
drivers/net/ethernet/microsoft/mana/mana_en.c | 4 ++--
drivers/net/ethernet/microsoft/mana/mana_ethtool.c | 2 +-
drivers/net/ethernet/microsoft/mana/shm_channel.c | 2 +-
{drivers/net/ethernet/microsoft => include/net}/mana/gdma.h | 0
.../net/ethernet/microsoft => include/net}/mana/hw_channel.h | 0
{drivers/net/ethernet/microsoft => include/net}/mana/mana.h | 0
.../ethernet/microsoft => include/net}/mana/mana_auxiliary.h | 0
.../net/ethernet/microsoft => include/net}/mana/shm_channel.h | 0
12 files changed, 9 insertions(+), 8 deletions(-)
rename {drivers/net/ethernet/microsoft => include/net}/mana/gdma.h (100%)
rename {drivers/net/ethernet/microsoft => include/net}/mana/hw_channel.h (100%)
rename {drivers/net/ethernet/microsoft => include/net}/mana/mana.h (100%)
rename {drivers/net/ethernet/microsoft => include/net}/mana/mana_auxiliary.h (100%)
rename {drivers/net/ethernet/microsoft => include/net}/mana/shm_channel.h (100%)MAINTAINERS +1 -0
@@ -9457,6 +9457,7 @@ F: include/asm-generic/hyperv-tlfs.h F: include/asm-generic/mshyperv.h F: include/clocksource/hyperv_timer.h F: include/linux/hyperv.h +F: include/net/mana F: include/uapi/linux/hyperv.h F: net/vmw_vsock/hyperv_transport.c F: tools/hv/
drivers/net/ethernet/microsoft/mana/gdma_main.c +1 -1
@@ -6,7 +6,7 @@#include <linux/utsname.h> #include <linux/version.h> -#include "mana.h" +#include <net/mana/mana.h> static u32 mana_gd_r32 (struct gdma_context *g,u64 offset) {
drivers/net/ethernet/microsoft/mana/hw_channel.c +2 -2
@@ -1,8 +1,8 @@// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause /* Copyright (c) 2021, Microsoft Corporation. */ -#include "gdma.h" -#include "hw_channel.h" +#include <net/mana/gdma.h> +#include <net/mana/hw_channel.h> static int mana_hwc_get_msg_index (struct hw_channel_context *hwc,u16 *msg_id) {
drivers/net/ethernet/microsoft/mana/mana_bpf.c +1 -1
@@ -8,7 +8,7 @@#include <linux/bpf_trace.h> #include <net/xdp.h> -#include "mana.h" +#include <net/mana/mana.h> void mana_xdp_tx (struct sk_buff *skb,struct net_device *ndev) {
drivers/net/ethernet/microsoft/mana/mana_en.c +2 -2
@@ -12,8 +12,8 @@#include <net/checksum.h> #include <net/ip6_checksum.h> -#include "mana.h" -#include "mana_auxiliary.h" +#include <net/mana/mana.h> +#include <net/mana/mana_auxiliary.h> static DEFINE_IDA (mana_adev_ida);
drivers/net/ethernet/microsoft/mana/mana_ethtool.c +1 -1
@@ -5,7 +5,7 @@#include <linux/etherdevice.h> #include <linux/ethtool.h> -#include "mana.h" +#include <net/mana/mana.h> static const struct { char name[ETH_GSTRING_LEN];
drivers/net/ethernet/microsoft/mana/shm_channel.c +1 -1
@@ -6,7 +6,7 @@#include <linux/io.h> #include <linux/mm.h> -#include "shm_channel.h" +#include <net/mana/shm_channel.h> #define PAGE_FRAME_L48_WIDTH_BYTES 6#define PAGE_FRAME_L48_WIDTH_BITS (PAGE_FRAME_L48_WIDTH_BYTES * 8)
include/net/mana/gdma.h renamed from drivers/net/ethernet/microsoft/mana/gdma.h +0 -0
include/net/mana/hw_channel.h renamed from drivers/net/ethernet/microsoft/mana/hw_channel.h +0 -0
include/net/mana/mana.h renamed from drivers/net/ethernet/microsoft/mana/mana.h +0 -0
include/net/mana/mana_auxiliary.h renamed from drivers/net/ethernet/microsoft/mana/mana_auxiliary.h +0 -0
include/net/mana/shm_channel.h renamed from drivers/net/ethernet/microsoft/mana/shm_channel.h +0 -0
Signature · 1 line
2.17.1
PATCH v9 9/12 net: mana: Define and process GDMA response code GDMA_STATUS_MORE_ENTRIES
In reply to [email protected]From: Ajay Sharma <[email protected]>
When doing memory registration, the PF may respond with GDMA_STATUS_MORE_ENTRIES to indicate a follow request is needed. This is not an error and should be processed as expected.
--- drivers/net/ethernet/microsoft/mana/hw_channel.c | 2 +- include/net/mana/gdma.h | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-)
drivers/net/ethernet/microsoft/mana/hw_channel.c +1 -1
@@ -836,7 +836,7 @@ int mana_hwc_send_request(struct hw_channel_context *hwc, u32 req_len, goto out; } -if (ctx->status_code) { +if (ctx->status_code && ctx->status_code !=GDMA_STATUS_MORE_ENTRIES ) {dev_err (hwc->dev,"HWC: Failed hw_channel req: 0x%x\n" , ctx->status_code); err = -EPROTO ;
include/net/mana/gdma.h +2 -0
@@ -9,6 +9,8 @@#include "shm_channel.h" +#define GDMA_STATUS_MORE_ENTRIES 0x00000105 +/* Structures labeled with "HW DATA" are exchanged with the hardware. All of * them are naturally aligned and hence don't need __packed. */
Signature · 1 line
2.17.1
PATCH v9 10/12 net: mana: Define data structures for allocating doorbell page from GDMA
In reply to [email protected]From: Long Li <[email protected]>
The RDMA device needs to allocate doorbell pages for each user context. Define the GDMA data structures for use by the RDMA driver.
---
Change log:
v4: use EXPORT_SYMBOL_NS
v7: move mana_gd_allocate_doorbell_page() and mana_gd_destroy_doorbell_page() to the RDMA driver
v8: remove extra line in header file
include/net/mana/gdma.h | 25 +++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
include/net/mana/gdma.h +24 -0
@@ -24,11 +24,15 @@ enum gdma_request_type {GDMA_GENERATE_TEST_EQE =10 ,GDMA_CREATE_QUEUE =12 ,GDMA_DISABLE_QUEUE =13 , +GDMA_ALLOCATE_RESOURCE_RANGE =22 , +GDMA_DESTROY_RESOURCE_RANGE =24 ,GDMA_CREATE_DMA_REGION =25 ,GDMA_DMA_REGION_ADD_PAGES =26 , GDMA_DESTROY_DMA_REGION = 27, }; +#define GDMA_RESOURCE_DOORBELL_PAGE 27 +enum gdma_queue_type {GDMA_INVALID_QUEUE , GDMA_SQ,
@@ -587,6 +591,26 @@ struct gdma_register_device_resp {u32 db_id; };/* HW DATA */ +struct gdma_allocate_resource_range_req { +struct gdma_req_hdr hdr; +u32 resource_type; +u32 num_resources; +u32 alignment; +u32 allocated_resources; +}; + +struct gdma_allocate_resource_range_resp { +struct gdma_resp_hdr hdr; +u32 allocated_resources; +}; + +struct gdma_destroy_resource_range_req { +struct gdma_req_hdr hdr; +u32 resource_type; +u32 num_resources; +u32 allocated_resources; +}; +/* GDMA_CREATE_QUEUE */ struct gdma_create_queue_req {struct gdma_req_hdr hdr;
Signature · 1 line
2.17.1
PATCH v9 12/12 RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter
In reply to [email protected]From: Long Li <[email protected]>
Add a RDMA VF driver for Microsoft Azure Network Adapter (MANA).
---
Change log:
v2:
Changed coding sytles/formats
Checked undersize for udata length
Changed all logging to use ibdev_xxx()
Avoided page array copy when doing MR
Sorted driver ops
Fixed warnings reported by kernel test robot <[email protected]>
v3:
More coding sytle/format changes
v4:
Process error on hardware vport configuration
v5:
Change licenses to GPL-2.0-only
Fix error handling in mana_ib_gd_create_dma_region()
v6:
rebased to rdma-next
removed redundant initialization to return value in mana_ib_probe() added missing tabs at the end of mana_ib_gd_create_dma_region()
v7:
move mana_gd_destroy_doorbell_page() and mana_gd_allocate_doorbell_page() from GDMA to this driver use ib_umem_find_best_pgsz() for finding page size for registering dma regions with hardware fix a bug that may double free mana_ind_table in mana_ib_create_qp_rss()
add Ajay Sharma <[email protected]> to maintainer list
add details to description in drivers/infiniband/hw/mana/Kconfig
change multiple lines comments to use RDMA style from NETDEV style
change mana_ib_dev_ops to static
use module_auxiliary_driver() in place of module_init and module_exit
move all user-triggerable error messages to debug messages
check for ind_tbl_size overflow in mana_ib_create_qp_rss()
v8:
instead of EFAULT, use return code from ib_copy_from_udata() fix the race condition on mana_cfg_vport()
use return code from mana_gd_allocate_doorbell_page() simplify error handling code in mana_ib_gd_create_dma_region() use U64_MAX in place of ((u64)(~(0ULL)))
remove confusing debug output on vport steering failure fix uninitialized cleanup index i in mana_ib_create_qp_rss()
v9:
define MANA_IB_MAX_MR for max number of supported MRs
MAINTAINERS | 9 + drivers/infiniband/Kconfig | 1 + drivers/infiniband/hw/Makefile | 1 + drivers/infiniband/hw/mana/Kconfig | 10 + drivers/infiniband/hw/mana/Makefile | 4 + drivers/infiniband/hw/mana/cq.c | 79 ++++ drivers/infiniband/hw/mana/device.c | 117 ++++++ drivers/infiniband/hw/mana/main.c | 507 ++++++++++++++++++++++++ drivers/infiniband/hw/mana/mana_ib.h | 162 ++++++++ drivers/infiniband/hw/mana/mr.c | 197 +++++++++ drivers/infiniband/hw/mana/qp.c | 506 +++++++++++++++++++++++ drivers/infiniband/hw/mana/wq.c | 115 ++++++ include/net/mana/mana.h | 3 + include/uapi/rdma/ib_user_ioctl_verbs.h | 1 + include/uapi/rdma/mana-abi.h | 66 +++ 15 files changed, 1778 insertions(+) create mode 100644 drivers/infiniband/hw/mana/Kconfig create mode 100644 drivers/infiniband/hw/mana/Makefile create mode 100644 drivers/infiniband/hw/mana/cq.c create mode 100644 drivers/infiniband/hw/mana/device.c create mode 100644 drivers/infiniband/hw/mana/main.c create mode 100644 drivers/infiniband/hw/mana/mana_ib.h create mode 100644 drivers/infiniband/hw/mana/mr.c create mode 100644 drivers/infiniband/hw/mana/qp.c create mode 100644 drivers/infiniband/hw/mana/wq.c create mode 100644 include/uapi/rdma/mana-abi.h
MAINTAINERS +9 -0
@@ -13506,6 +13506,15 @@ F: drivers/scsi/smartpqi/smartpqi*.[ch] F: include/linux/cciss*.h F: include/uapi/linux/cciss*.h +MICROSOFT MANA RDMA DRIVER +M: Long Li <[email protected]> +M: Ajay Sharma <[email protected]> +L: [email protected] +S: Supported +F: drivers/infiniband/hw/mana/ +F: include/net/mana +F: include/uapi/rdma/mana-abi.h + MICROSOFT SURFACE AGGREGATOR TABLET-MODE SWITCH M: Maximilian Luz <[email protected]> L: [email protected]
drivers/infiniband/Kconfig +1 -0
@@ -85,6 +85,7 @@ source "drivers/infiniband/hw/erdma/Kconfig" source "drivers/infiniband/hw/hfi1/Kconfig" source "drivers/infiniband/hw/hns/Kconfig" source "drivers/infiniband/hw/irdma/Kconfig" +source "drivers/infiniband/hw/mana/Kconfig" source "drivers/infiniband/hw/mlx4/Kconfig" source "drivers/infiniband/hw/mlx5/Kconfig" source "drivers/infiniband/hw/mthca/Kconfig"
drivers/infiniband/hw/Makefile +1 -0
@@ -4,6 +4,7 @@ obj-$(CONFIG_INFINIBAND_QIB) += qib/ obj-$(CONFIG_INFINIBAND_CXGB4) += cxgb4/ obj-$(CONFIG_INFINIBAND_EFA) += efa/ obj-$(CONFIG_INFINIBAND_IRDMA) += irdma/ +obj-$(CONFIG_MANA_INFINIBAND) += mana/ obj-$(CONFIG_MLX4_INFINIBAND) += mlx4/ obj-$(CONFIG_MLX5_INFINIBAND) += mlx5/ obj-$(CONFIG_INFINIBAND_OCRDMA) += ocrdma/
drivers/infiniband/hw/mana/Kconfig new file +10 -0
@@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0-only +config MANA_INFINIBAND + tristate "Microsoft Azure Network Adapter support" + depends on NETDEVICES && ETHERNET && PCI && MICROSOFT_MANA + help + This driver provides low-level RDMA support for Microsoft Azure + Network Adapter (MANA). MANA supports RDMA features that can be used + for workloads (e.g. DPDK, MPI etc) that uses RDMA verbs to directly + access hardware from user-mode processes in Microsoft Azure cloud + environment.
drivers/infiniband/hw/mana/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_MANA_INFINIBAND) += mana_ib.o + +mana_ib-y := device.o main.o wq.o qp.o cq.o mr.o
drivers/infiniband/hw/mana/cq.c new file +79 -0
79 lines not on this page · raw message
drivers/infiniband/hw/mana/device.c new file +117 -0
117 lines not on this page · raw message
drivers/infiniband/hw/mana/main.c new file +507 -0
507 lines not on this page · raw message
drivers/infiniband/hw/mana/mana_ib.h new file +162 -0
162 lines not on this page · raw message
drivers/infiniband/hw/mana/mr.c new file +197 -0
197 lines not on this page · raw message
drivers/infiniband/hw/mana/qp.c new file +506 -0
506 lines not on this page · raw message
drivers/infiniband/hw/mana/wq.c new file +115 -0
115 lines not on this page · raw message
include/net/mana/mana.h +3 -0
@@ -412,6 +412,9 @@ int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf);extern const struct ethtool_ops mana_ethtool_ops; +/* A CQ can be created not associated with any EQ */ +#define GDMA_CQ_NO_EQ 0xffff +struct mana_obj_spec {u32 queue_index;u64 gdma_region;
include/uapi/rdma/ib_user_ioctl_verbs.h +1 -0
@@ -251,6 +251,7 @@ enum rdma_driver_id {RDMA_DRIVER_EFA ,RDMA_DRIVER_SIW ,RDMA_DRIVER_ERDMA , + RDMA_DRIVER_MANA, }; enum ib_uverbs_gid_type {
include/uapi/rdma/mana-abi.h new file +66 -0
66 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v9 11/12 net: mana: Define data structures for protection domain and memory registration
In reply to [email protected]From: Ajay Sharma <[email protected]>
The MANA hardware support protection domain and memory registration for use in RDMA environment. Add those definitions and expose them for use by the RDMA driver.
---
Change log:
v3: format/coding style changes
v5: remove unused code dealing with kernel-mode MR
.../net/ethernet/microsoft/mana/gdma_main.c | 27 ++-- drivers/net/ethernet/microsoft/mana/mana_en.c | 18 +-- include/net/mana/gdma.h | 121 +++++++++++++++++- 3 files changed, 143 insertions(+), 23 deletions(-)
drivers/net/ethernet/microsoft/mana/gdma_main.c +17 -10
@@ -198,7 +198,7 @@ static int mana_gd_create_hw_eq(struct gdma_context *gc, req.type = queue->type; req.pdid = queue->gdma_dev->pdid; req.doolbell_id = queue->gdma_dev->doorbell; - req.gdma_region = queue->mem_info.gdma_region; + req.gdma_region = queue->mem_info.dma_region_handle; req.queue_size = queue->queue_size; req.log2_throttle_limit = queue->eq.log2_throttle_limit; req.eq_pci_msix_index = queue->eq.msix_index;
@@ -212,7 +212,7 @@ static int mana_gd_create_hw_eq(struct gdma_context *gc, queue->id = resp.queue_index; queue->eq.disable_needed = true; - queue->mem_info.gdma_region =GDMA_INVALID_DMA_REGION ; + queue->mem_info.dma_region_handle =GDMA_INVALID_DMA_REGION ;return 0 ; }
@@ -666,24 +666,30 @@ int mana_gd_create_hwc_queue(struct gdma_dev *gd,return err; } -static void mana_gd_destroy_dma_region (struct gdma_context *gc,u64 gdma_region) +int mana_gd_destroy_dma_region (struct gdma_context *gc, +gdma_obj_handle_t dma_region_handle) {struct gdma_destroy_dma_region_req req = {};struct gdma_general_resp resp = {};int err; -if (gdma_region ==GDMA_INVALID_DMA_REGION ) -return ; +if (dma_region_handle ==GDMA_INVALID_DMA_REGION ) +return 0 ;mana_gd_init_req_hdr (&req.hdr,GDMA_DESTROY_DMA_REGION ,sizeof (req),sizeof (resp)); - req.gdma_region = gdma_region; + req.dma_region_handle = dma_region_handle; err =mana_gd_send_request (gc,sizeof (req), &req,sizeof (resp), &resp); -if (err || resp.hdr.status) +if (err || resp.hdr.status) {dev_err (gc->dev,"Failed to destroy DMA region: %d, 0x%x\n" , err, resp.hdr.status); +return -EPROTO ; + } + +return 0 ; } +EXPORT_SYMBOL_NS (mana_gd_destroy_dma_region,NET_MANA );static int mana_gd_create_dma_region (struct gdma_dev *gd,struct gdma_mem_info *gmi)
@@ -728,14 +734,15 @@ static int mana_gd_create_dma_region(struct gdma_dev *gd,if (err) goto out; -if (resp.hdr.status || resp.gdma_region ==GDMA_INVALID_DMA_REGION ) { +if (resp.hdr.status || + resp.dma_region_handle ==GDMA_INVALID_DMA_REGION ) {dev_err (gc->dev,"Failed to create DMA region: 0x%x\n" , resp.hdr.status); err = -EPROTO ; goto out; } - gmi->gdma_region = resp.gdma_region; + gmi->dma_region_handle = resp.dma_region_handle; out:kfree (req);return err;
@@ -858,7 +865,7 @@ void mana_gd_destroy_queue(struct gdma_context *gc, struct gdma_queue *queue)return ; } -mana_gd_destroy_dma_region (gc, gmi->gdma_region); +mana_gd_destroy_dma_region (gc, gmi->dma_region_handle);mana_gd_free_memory (gmi);kfree (queue); }
drivers/net/ethernet/microsoft/mana/mana_en.c +10 -8
@@ -1523,10 +1523,10 @@ static int mana_create_txq(struct mana_port_context *apc,memset (&wq_spec,0 ,sizeof (wq_spec));memset (&cq_spec,0 ,sizeof (cq_spec)); - wq_spec.gdma_region = txq->gdma_sq->mem_info.gdma_region; + wq_spec.gdma_region = txq->gdma_sq->mem_info.dma_region_handle; wq_spec.queue_size = txq->gdma_sq->queue_size; - cq_spec.gdma_region = cq->gdma_cq->mem_info.gdma_region; + cq_spec.gdma_region = cq->gdma_cq->mem_info.dma_region_handle; cq_spec.queue_size = cq->gdma_cq->queue_size; cq_spec.modr_ctx_id =0 ; cq_spec.attached_eq = cq->gdma_cq->cq.parent->id;
@@ -1541,8 +1541,10 @@ static int mana_create_txq(struct mana_port_context *apc, txq->gdma_sq->id = wq_spec.queue_index; cq->gdma_cq->id = cq_spec.queue_index; - txq->gdma_sq->mem_info.gdma_region =GDMA_INVALID_DMA_REGION ; - cq->gdma_cq->mem_info.gdma_region =GDMA_INVALID_DMA_REGION ; + txq->gdma_sq->mem_info.dma_region_handle = +GDMA_INVALID_DMA_REGION ; + cq->gdma_cq->mem_info.dma_region_handle = +GDMA_INVALID_DMA_REGION ; txq->gdma_txq_id = txq->gdma_sq->id;
@@ -1753,10 +1755,10 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,memset (&wq_spec,0 ,sizeof (wq_spec));memset (&cq_spec,0 ,sizeof (cq_spec)); - wq_spec.gdma_region = rxq->gdma_rq->mem_info.gdma_region; + wq_spec.gdma_region = rxq->gdma_rq->mem_info.dma_region_handle; wq_spec.queue_size = rxq->gdma_rq->queue_size; - cq_spec.gdma_region = cq->gdma_cq->mem_info.gdma_region; + cq_spec.gdma_region = cq->gdma_cq->mem_info.dma_region_handle; cq_spec.queue_size = cq->gdma_cq->queue_size; cq_spec.modr_ctx_id =0 ; cq_spec.attached_eq = cq->gdma_cq->cq.parent->id;
@@ -1769,8 +1771,8 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc, rxq->gdma_rq->id = wq_spec.queue_index; cq->gdma_cq->id = cq_spec.queue_index; - rxq->gdma_rq->mem_info.gdma_region =GDMA_INVALID_DMA_REGION ; - cq->gdma_cq->mem_info.gdma_region =GDMA_INVALID_DMA_REGION ; + rxq->gdma_rq->mem_info.dma_region_handle =GDMA_INVALID_DMA_REGION ; + cq->gdma_cq->mem_info.dma_region_handle =GDMA_INVALID_DMA_REGION ; rxq->gdma_id = rxq->gdma_rq->id; cq->gdma_id = cq->gdma_cq->id;
include/net/mana/gdma.h +116 -5
175 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v9 8/12 net: mana: Define max values for SGL entries
In reply to [email protected]From: Long Li <[email protected]>
The number of maximum SGl entries should be computed from the maximum WQE size for the intended queue type and the corresponding OOB data size. This guarantees the hardware queue can successfully queue requests up to the queue depth exposed to the upper layer.
--- drivers/net/ethernet/microsoft/mana/mana_en.c | 2 +- include/net/mana/gdma.h | 7 +++++++ include/net/mana/mana.h | 4 +--- 3 files changed, 9 insertions(+), 4 deletions(-)
drivers/net/ethernet/microsoft/mana/mana_en.c +1 -1
@@ -189,7 +189,7 @@ int mana_start_xmit(struct sk_buff *skb, struct net_device *ndev) pkg.wqe_req.client_data_unit =0 ; pkg.wqe_req.num_sge =1 +skb_shinfo (skb)->nr_frags; -WARN_ON_ONCE (pkg.wqe_req.num_sge >30 ); +WARN_ON_ONCE (pkg.wqe_req.num_sge >MAX_TX_WQE_SGL_ENTRIES );if (pkg.wqe_req.num_sge <=ARRAY_SIZE (pkg.sgl_array)) { pkg.wqe_req.sgl = pkg.sgl_array;
include/net/mana/gdma.h +7 -0
@@ -427,6 +427,13 @@ struct gdma_wqe {#define MAX_TX_WQE_SIZE 512#define MAX_RX_WQE_SIZE 256 +#define MAX_TX_WQE_SGL_ENTRIES ((GDMA_MAX_SQE_SIZE - \ + sizeof(struct gdma_sge) - INLINE_OOB_SMALL_SIZE) / \ + sizeof(struct gdma_sge)) + +#define MAX_RX_WQE_SGL_ENTRIES ((GDMA_MAX_RQE_SIZE - \ + sizeof(struct gdma_sge)) / sizeof(struct gdma_sge)) +struct gdma_cqe {u32 cqe_data[GDMA_COMP_DATA_SIZE /4 ];
include/net/mana/mana.h +1 -3
@@ -265,8 +265,6 @@ struct mana_cq {int budget; }; -#define GDMA_MAX_RQE_SGES 15 -struct mana_recv_buf_oob {/* A valid GDMA work request representing the data buffer. */ struct gdma_wqe_request wqe_req;
@@ -276,7 +274,7 @@ struct mana_recv_buf_oob {/* SGL of the buffer going to be sent has part of the work request. */ u32 num_sge; -struct gdma_sge sgl[GDMA_MAX_RQE_SGES ]; +struct gdma_sge sgl[MAX_RX_WQE_SGL_ENTRIES ];/* Required to store the result of mana_gd_post_work_request. * gdma_posted_wqe_info.wqe_size_in_bu is required for progressing the
Signature · 1 line
2.17.1
Re: [Patch v9 03/12] net: mana: Handle vport sharing between devices
In reply to [email protected]Quoted 102 lines · On 2022/10/22 8:01, [email protected] wrote:
From: Long Li <[email protected]>
For outgoing packets, the PF requires the VF to configure the vport with corresponding protection domain and doorbell ID for the kernel or user context. The vport can't be shared between different contexts.
Implement the logic to exclusively take over the vport by either the Ethernet device or RDMA device.
Reviewed-by Dexuan Cui <[email protected]>Signed-off-by Long Li <[email protected]>Acked-by Haiyang Zhang <[email protected]>---
Change log:
v2: use refcount instead of directly using atomic variables
v4: change to mutex to avoid possible race with refcount
v5: add detailed comments explaining vport sharing, useEXPORT_SYMBOL_NS
v6: rebased to rdma-nextdrivers/net/ethernet/microsoft/mana/mana.h | 7 +++ drivers/net/ethernet/microsoft/mana/mana_en.c | 53 ++++++++++++++++++- 2 files changed, 58 insertions(+), 2 deletions(-)drivers/net/ethernet/microsoft/mana/mana.h +7
-0@@ -380,6 +380,10 @@ struct mana_port_context {mana_handle_t port_handle;mana_handle_t pf_filter_handle; +/* Mutex for sharing access to vport_use_count */ +struct mutex vport_mutex; +int vport_use_count; +u16 port_idx;bool port_is_up;@@ -631,4 +635,7 @@ struct mana_tx_package { struct gdma_posted_wqe_info wqe_info; }; +int mana_cfg_vport (struct mana_port_context *apc,u32 protection_dom_id, +u32 doorbell_pg_id); +void mana_uncfg_vport (struct mana_port_context *apc);#endif /* _MANA_H */ drivers/net/ethernet/microsoft/mana/mana_en.c +37
-2@@ -646,13 +646,48 @@ static int mana_query_vport_cfg(struct mana_port_context *apc, u32 vport_index,return 0 ; } -static int mana_cfg_vport (struct mana_port_context *apc,u32 protection_dom_id, -u32 doorbell_pg_id) +void mana_uncfg_vport (struct mana_port_context *apc) +{ +mutex_lock (&apc->vport_mutex); + apc->vport_use_count--; +WARN_ON (apc->vport_use_count <0 ); +mutex_unlock (&apc->vport_mutex); +} +EXPORT_SYMBOL_NS (mana_uncfg_vport,NET_MANA ); + +int mana_cfg_vport (struct mana_port_context *apc,u32 protection_dom_id, +u32 doorbell_pg_id) {struct mana_config_vport_resp resp = {};struct mana_config_vport_req req = {};int err; +/* This function is used to program the Ethernet port in the hardware +* table. It can be called from the Ethernet driver or the RDMA driver. +* +* For Ethernet usage, the hardware supports only one active user on a +* physical port. The driver checks on the port usage before programming +* the hardware when creating the RAW QP (RDMA driver) or exposing the +* device to kernel NET layer (Ethernet driver). +* +* Because the RDMA driver doesn't know in advance which QP type the +* user will create, it exposes the device with all its ports. The user +* may not be able to create RAW QP on a port if this port is already +* in used by the Ethernet driver from the kernel. +* +* This physical port limitation only applies to the RAW QP. For RC QP, +* the hardware doesn't have this limitation. The user can create RC +* QPs on a physical port up to the hardware limits independent of the +* Ethernet usage on the same port. +*/ +mutex_lock (&apc->vport_mutex); +if (apc->vport_use_count >0 ) { +mutex_unlock (&apc->vport_mutex); +return -EBUSY ; + } + apc->vport_use_count++; +mutex_unlock (&apc->vport_mutex); +mana_gd_init_req_hdr (&req.hdr,MANA_CONFIG_VPORT_TX ,sizeof (req),sizeof (resp)); req.vport = apc->port_handle;
drivers/net/ethernet/microsoft/mana/mana_en.c static int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, · line 722
Quoted 9 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +5
-0@@ -679,9 +714,16 @@ static int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, apc->tx_shortform_allowed = resp.short_form_allowed; apc->tx_vp_offset = resp.tx_vport_offset; + +netdev_info (apc->ndev,"Configured vPort %llu PD %u DB %u\n" , + apc->port_handle, protection_dom_id, doorbell_pg_id); out: +if (err) +mana_uncfg_vport (apc);
There seems to be a similar race between error handling here and the "apc->vport_use_count > 0" checking above as pointed out in v7.
Quoted 36 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +9
-0+return err; } +EXPORT_SYMBOL_NS (mana_cfg_vport,NET_MANA ); static int mana_cfg_vport_steering(struct mana_port_context *apc, enum TRI_STATE rx,@@ -742,6 +784,9 @@ static int mana_cfg_vport_steering(struct mana_port_context *apc, resp.hdr.status); err = -EPROTO ; } + +netdev_info (ndev,"Configured steering vPort %llu entries %u\n" , + apc->port_handle, num_entries); out:kfree (req);return err;@@ -1804,6 +1849,7 @@ static void mana_destroy_vport(struct mana_port_context *apc) }mana_destroy_txq (apc); +mana_uncfg_vport (apc);if (gd->gdma_context->is_pf)mana_pf_deregister_hw_vport (apc);@@ -2076,6 +2122,9 @@ static int mana_probe_port(struct mana_context *ac, int port_idx, apc->pf_filter_handle =INVALID_MANA_HANDLE ; apc->port_idx = port_idx; +mutex_init (&apc->vport_mutex); + apc->vport_use_count =0 ; + ndev->netdev_ops = &mana_devops; ndev->ethtool_ops = &mana_ethtool_ops; ndev->mtu =ETH_DATA_LEN ;
RE: [Patch v9 03/12] net: mana: Handle vport sharing between devices
Quoted 45 lines
Quoted 1 line
+int mana_cfg_vport(struct
mana_port_context*apc, u32
protection_dom_id,drivers/net/ethernet/microsoft/mana/mana_en.c static int mana_query_vport_cfg(struct mana_port_context *apc, u32 vport_index,
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +5
-0+ u32 doorbell_pg_id) {struct mana_config_vport_resp resp = {};struct mana_config_vport_req req = {};int err; +/* This function is used to program the Ethernet port in the hardware +* table. It can be called from the Ethernet driver or the RDMA driver. +* +* For Ethernet usage, the hardware supports only one active user on a
Review
Quoted 1 line
(quoted diff) +1
-0+ * physical port. The driver checks on the port usage beforeprogramming
Review
Quoted 1 line
(quoted diff) +1
-0+ * the hardware when creating the RAW QP (RDMA driver) orexposing the
drivers/net/ethernet/microsoft/mana/mana_en.c static int mana_query_vport_cfg(struct mana_port_context *apc, u32 vport_index,
Quoted 3 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +3
-0+* device to kernel NET layer (Ethernet driver). +* +* Because the RDMA driver doesn't know in advance which QP type the
drivers/net/ethernet/microsoft/mana/mana_en.c static int mana_query_vport_cfg(struct mana_port_context *apc, u32 vport_index,
Quoted 7 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +7
-0+* user will create, it exposes the device with all its ports. The user +* may not be able to create RAW QP on a port if this port is already +* in used by the Ethernet driver from the kernel. +* +* This physical port limitation only applies to the RAW QP. For RC QP, +* the hardware doesn't have this limitation. The user can create RC +* QPs on a physical port up to the hardware limits independent of the
Quoted 16 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +10
-0+* Ethernet usage on the same port. +*/ +mutex_lock (&apc->vport_mutex); +if (apc->vport_use_count >0 ) { +mutex_unlock (&apc->vport_mutex); +return -EBUSY ; + } + apc->vport_use_count++; +mutex_unlock (&apc->vport_mutex); +mana_gd_init_req_hdr (&req.hdr,MANA_CONFIG_VPORT_TX ,sizeof (req),sizeof (resp)); req.vport = apc->port_handle;@@ -679,9 +714,16 @@ static int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id,
Quoted 11 lines
drivers/net/ethernet/microsoft/mana/mana_en.c static int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, · line 722
Quoted 8 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +5
-0apc->tx_shortform_allowed = resp.short_form_allowed; apc->tx_vp_offset = resp.tx_vport_offset; + +netdev_info (apc->ndev,"Configured vPort %llu PD %u DB %u\n" , + apc->port_handle, protection_dom_id, doorbell_pg_id); out: +if (err) +mana_uncfg_vport (apc);There seems to be a similar race between error handling here and the "apc-
Quoted 1 line
vport_use_count> 0" checking above as pointed out in v7.
Thanks for looking into this.
This is different to the locking bug in mana_ib_cfg_vport(). The vport sharing between Ethernet and RDMA is exclusive, not shared. If another driver tries to take the vport while it is being configured, it will fail immediately. It is by design to prevent possible deadlock.
Re: [Patch v9 03/12] net: mana: Handle vport sharing between devices
Quoted 55 lines · Long Li
Quoted 55 lines
Quoted 1 line
+int mana_cfg_vport(struct
mana_port_context*apc, u32
protection_dom_id,drivers/net/ethernet/microsoft/mana/mana_en.c static int mana_query_vport_cfg(struct mana_port_context *apc, u32 vport_index,
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +5
-0+ u32 doorbell_pg_id) {struct mana_config_vport_resp resp = {};struct mana_config_vport_req req = {};int err; +/* This function is used to program the Ethernet port in the hardware +* table. It can be called from the Ethernet driver or the RDMA driver. +* +* For Ethernet usage, the hardware supports only one active user on a
Review
Quoted 1 line
(quoted diff) +1
-0+ * physical port. The driver checks on the port usage beforeprogramming
Review
Quoted 1 line
(quoted diff) +1
-0+ * the hardware when creating the RAW QP (RDMA driver) orexposing the
drivers/net/ethernet/microsoft/mana/mana_en.c static int mana_query_vport_cfg(struct mana_port_context *apc, u32 vport_index,
Quoted 3 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +3
-0+* device to kernel NET layer (Ethernet driver). +* +* Because the RDMA driver doesn't know in advance which QP type the
drivers/net/ethernet/microsoft/mana/mana_en.c static int mana_query_vport_cfg(struct mana_port_context *apc, u32 vport_index,
Quoted 7 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +7
-0+* user will create, it exposes the device with all its ports. The user +* may not be able to create RAW QP on a port if this port is already +* in used by the Ethernet driver from the kernel. +* +* This physical port limitation only applies to the RAW QP. For RC QP, +* the hardware doesn't have this limitation. The user can create RC +* QPs on a physical port up to the hardware limits independent of the
Quoted 14 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +10
-0+* Ethernet usage on the same port. +*/ +mutex_lock (&apc->vport_mutex); +if (apc->vport_use_count >0 ) { +mutex_unlock (&apc->vport_mutex); +return -EBUSY ; + } + apc->vport_use_count++; +mutex_unlock (&apc->vport_mutex); +mana_gd_init_req_hdr (&req.hdr,MANA_CONFIG_VPORT_TX ,sizeof (req),sizeof (resp)); req.vport = apc->port_handle;drivers/net/ethernet/microsoft/mana/mana_en.c static int mana_cfg_vport(struct · line 723
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +5
-0@@ -679,9 +714,16 @@ static int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, apc->tx_shortform_allowed = resp.short_form_allowed; apc->tx_vp_offset = resp.tx_vport_offset; + +netdev_info (apc->ndev,"Configured vPort %llu PD %u DB %u\n" , + apc->port_handle, protection_dom_id, doorbell_pg_id); out: +if (err) +mana_uncfg_vport (apc);There seems to be a similar race between error handling here and the "apc-
Quoted 7 lines
Quoted 1 line
Quoted 1 line
vport_use_count> 0" checking above as pointed out in v7.Thanks for looking into this.
This is different to the locking bug in
mana_ib_cfg_vport(). The vport sharing between Ethernet and RDMA is exclusive, not shared. If another driver tries to take the vport while it is being configured, it will fail immediately. It is by
Suppose the following steps:
1. Ethernet driver take the lock first and do a "apc->vport_use_count++", and release the lock;
2. RDMA driver take the lock, do "apc->vport_use_count > 0" checking and return -EBUSY;
3. mana_send_request() or mana_verify_resp_hdr() return error to Ethernet driver.
It seems that vport is left unused when above happens, if that is what you wanted?
Quoted 1 line
design to prevent possible deadlock.
I am not sure I understand the deadlock here.
Quoted 1 line
RE: [Patch v9 03/12] net: mana: Handle vport sharing between devices
Quoted 33 lines
Quoted 14 lines
Quoted 14 lines
drivers/net/ethernet/microsoft/mana/mana_en.c static int mana_cfg_vport(struct · line 723
Quoted 11 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +5
-0@@ -679,9 +714,16 @@ static int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, apc->tx_shortform_allowed = resp.short_form_allowed; apc->tx_vp_offset = resp.tx_vport_offset; + +netdev_info (apc->ndev,"Configured vPort %llu PD %u DB %u\n" , + apc->port_handle, protection_dom_id, doorbell_pg_id); out: +if (err) +mana_uncfg_vport (apc);There seems to be a similar race between error handling here and the "apc-
Quoted 8 lines
Quoted 1 line
Quoted 1 line
vport_use_count> 0" checking above as pointed out in v7.Thanks for looking into this.
This is different to the locking bug in
mana_ib_cfg_vport(). The vport sharing between Ethernet and RDMA is exclusive, not shared. If another driver tries to take the vport while it is being configured, it will fail immediately. It is bySuppose the following steps:
1. Ethernet driver take the lock first and do a "apc->vport_use_count++", and
release the lock;
2. RDMA driver take the lock, do "apc->vport_use_count> 0" checking and return
-EBUSY;
3.mana_send_request()ormana_verify_resp_hdr()return error to Ethernet driver.
Quoted 2 lines
It seems that vport is left unused when above happens, if that is what you wanted?
Yes, in this case the vport is left unused. There is no resource leak. This is expected.
Quoted 5 lines
Quoted 3 lines
design to prevent possible deadlock.
I am not sure I understand the deadlock here.
Because we are dealing with two drivers. I don't want to block as mana_send_request() is a blocking call.
Jason Gunthorpe
maintainerRe: [Patch v9 12/12] RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter
In reply to [email protected]Quoted 15 lines · On Fri, Oct 21, 2022 at 05:01:29PM -0700, [email protected] wrote:
drivers/infiniband/hw/mana/main.c new file +15
-0+int mana_ib_gd_create_dma_region (struct mana_ib_dev *dev,struct ib_umem *umem, +mana_handle_t *gdma_region) +{ +struct gdma_dma_region_add_pages_req *add_req =NULL ; +struct gdma_create_dma_region_resp create_resp = {}; +struct gdma_create_dma_region_req *create_req; +size_t num_pages_cur, num_pages_to_handle; +unsigned int create_req_msg_size; +struct hw_channel_context *hwc; +struct ib_block_iter biter; +size_t max_pgs_create_cmd; +struct gdma_context *gc; +size_t num_pages_total; +struct gdma_dev *mdev; +unsigned long page_sz;
drivers/infiniband/hw/mana/main.c line 272
Quoted 10 lines
drivers/infiniband/hw/mana/main.c new file +10
-0+void *request_buf; +unsigned int i; +int err; + + mdev = dev->gdma_dev; + gc = mdev->gdma_context; + hwc = gc->hwc.driver_data; + +/* Hardware requires dma region to align to chosen page size */ + page_sz =ib_umem_find_best_pgsz (umem,PAGE_SZ_BM ,0 );
Does your HW support arbitary MR offsets in the IOVA?
struct ib_mr *mana_ib_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 length,
u64 iova, int access_flags, struct ib_udata *udata)
{
[..]
err = mana_ib_gd_create_dma_region(dev, mr->umem,&dma_region_handle); .. mr_params.gva.virtual_address = iova;
Eg if I set iova to 1 and length to PAGE_SIZE and pass in a umem which is fully page aligned, will the HW work, or will it DMA to the wrong locations?
All other RDMA HW requires passing iova to the ib_umem_find_best_pgsz() specifically to reject/adjust the misalignment of the IOVA relative to the selected pagesize.
drivers/infiniband/hw/mana/main.c line 318
Quoted 10 lines
drivers/infiniband/hw/mana/main.c new file +10
-0+__rdma_umem_block_iter_start (&biter, umem, page_sz); + +for (i =0 ; i < num_pages_to_handle; ++i) { +dma_addr_t cur_addr; + +__rdma_block_iter_next (&biter); + cur_addr =rdma_block_iter_dma_address (&biter); + + create_req->page_addr_list[i] = cur_addr; + }
This loop is still a mess, why can you not write it as I said for v6?
Usually the way these loops are structured is to fill the array and then check for fullness, trigger an action to drain the array, and reset the indexes back to the start.
so do the usual
rdma_umem_for_each_dma_block() {
page_addr_list[tail++] = rdma_block_iter_dma_address(&biter); if (tail >= num_pages_to_handle) {
mana_gd_send_request()
reset buffer
tail = 0
}
}
if (tail)
mana_gd_send_request()
Jason
RE: [Patch v9 12/12] RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter
Quoted 20 lines
Quoted 1 line
+int mana_ib_gd_create_dma_region(struct
mana_ib_dev*dev, struct
ib_umem*umem,Quoted 18 lines
drivers/infiniband/hw/mana/main.c new file +18
-0+ mana_handle_t *gdma_region) +{ +struct gdma_dma_region_add_pages_req *add_req =NULL ; +struct gdma_create_dma_region_resp create_resp = {}; +struct gdma_create_dma_region_req *create_req; +size_t num_pages_cur, num_pages_to_handle; +unsigned int create_req_msg_size; +struct hw_channel_context *hwc; +struct ib_block_iter biter; +size_t max_pgs_create_cmd; +struct gdma_context *gc; +size_t num_pages_total; +struct gdma_dev *mdev; +unsigned long page_sz; +void *request_buf; +unsigned int i; +int err; +
Quoted 8 lines
drivers/infiniband/hw/mana/main.c line 272
Quoted 6 lines
drivers/infiniband/hw/mana/main.c new file +6
-0+ mdev = dev->gdma_dev; + gc = mdev->gdma_context; + hwc = gc->hwc.driver_data; + +/* Hardware requires dma region to align to chosen page size */ + page_sz =ib_umem_find_best_pgsz (umem,PAGE_SZ_BM ,0 );Does your HW support arbitary MR offsets in the IOVA?
Yes, the HW supports arbitrary MR offsets. I'm checking with hardware guys to confirm.
Quoted 46 lines
struct
ib_mr*mana_ib_reg_user_mr(structib_pd*ibpd, u64 start, u64 length,
u64 iova, intaccess_flags, structib_udata*udata)
{
[..]err = mana_ib_gd_create_dma_region(dev, mr-
Quoted 1 line
umem,&dma_region_handle);
..
mr_params.gva.virtual_address = iova;Eg if I set iova to 1 and length to
PAGE_SIZEand pass in a umem which is fully page aligned, will the HW work, or will it DMA to the wrong locations?All other RDMA HW requires passing iova to the
ib_umem_find_best_pgsz()specifically to reject/adjust the misalignment of the IOVA relative to the selected pagesize.drivers/infiniband/hw/mana/main.c line 318
Quoted 10 lines
drivers/infiniband/hw/mana/main.c new file +10
-0+__rdma_umem_block_iter_start (&biter, umem, page_sz); + +for (i =0 ; i < num_pages_to_handle; ++i) { +dma_addr_t cur_addr; + +__rdma_block_iter_next (&biter); + cur_addr =rdma_block_iter_dma_address (&biter); + + create_req->page_addr_list[i] = cur_addr; + }This loop is still a mess, why can you not write it as I said for v6?
Usually the way these loops are structured is to fill the array and then check
for fullness, trigger an action to drain the array, and reset the indexes back to the start.so do the usual
rdma_umem_for_each_dma_block() { page_addr_list[tail++] = rdma_block_iter_dma_address(&biter); if (tail >= num_pages_to_handle) { mana_gd_send_request() reset buffer tail = 0 }
Quoted 4 lines
}
if (tail)
mana_gd_send_request()
I tried to recode this section; the new code looks like the following. It's 30 lines longer than the previous version.
The difficulty is that there are two PF messages involved: the 1st one is for creating the initial pages, the 2nd (and subsequent) ones are for adding more pages. So going through the page sequence as seen from the hardware side makes it shorter.
What do you think of this version?
static int
mana_ib_gd_first_dma_region(struct mana_ib_dev *dev,
struct gdma_context *gc,
struct gdma_create_dma_region_req *create_req, size_t num_pages, mana_handle_t *gdma_region)
{
struct gdma_create_dma_region_resp create_resp = {};
unsigned int create_req_msg_size;
int err;
create_req_msg_size =
struct_size(create_req, page_addr_list, num_pages);
create_req->page_addr_list_len = num_pages;
err = mana_gd_send_request(gc, create_req_msg_size, create_req,
sizeof(create_resp), &create_resp);
if (err || create_resp.hdr.status) {
ibdev_dbg(&dev->ib_dev,
"Failed to create DMA region: %d, 0x%x\n",
err, create_resp.hdr.status);
if (!err)
err = -EPROTO;
return err;
}
*gdma_region = create_resp.dma_region_handle;
ibdev_dbg(&dev->ib_dev, "Created DMA region handle 0x%llx\n",
*gdma_region);
return 0;
}static int mana_ib_gd_add_dma_region(struct mana_ib_dev *dev,
struct gdma_context *gc,
struct gdma_dma_region_add_pages_req *add_req, unsigned int num_pages, u32 expected_status)
{
unsigned int add_req_msg_size =
struct_size(add_req, page_addr_list, num_pages);
struct gdma_general_resp add_resp = {};
int err;
mana_gd_init_req_hdr(&add_req->hdr, GDMA_DMA_REGION_ADD_PAGES,
add_req_msg_size, sizeof(add_resp));
add_req->page_addr_list_len = num_pages;
err = mana_gd_send_request(gc, add_req_msg_size, add_req,
sizeof(add_resp), &add_resp);
if (err || add_resp.hdr.status != expected_status) {
ibdev_dbg(&dev->ib_dev,
"Failed to create DMA region: %d, 0x%x\n",
err, add_resp.hdr.status);
if (!err)
err = -EPROTO;
return err;
}
return 0;
}int mana_ib_gd_create_dma_region(struct mana_ib_dev *dev, struct ib_umem *umem,
mana_handle_t *gdma_region)
{
struct gdma_dma_region_add_pages_req *add_req = NULL;
size_t num_pages_processed = 0, num_pages_to_handle;
struct gdma_create_dma_region_resp create_resp = {};
struct gdma_create_dma_region_req *create_req;
unsigned int create_req_msg_size;
struct hw_channel_context *hwc;
size_t max_pgs_create_cmd;
size_t max_pgs_add_cmd;
struct ib_block_iter biter;
struct gdma_context *gc;
size_t num_pages_total;
struct gdma_dev *mdev;
unsigned long page_sz;
unsigned int tail = 0;
u64 *page_addr_list;
void *request_buf;
int err;
mdev = dev->gdma_dev;
gc = mdev->gdma_context;
hwc = gc->hwc.driver_data;
/* Hardware requires dma region to align to chosen page size */
page_sz = ib_umem_find_best_pgsz(umem, PAGE_SZ_BM, 0);
if (!page_sz) {
ibdev_dbg(&dev->ib_dev, "failed to find page size.\n");
return -ENOMEM;
}
num_pages_total = ib_umem_num_dma_blocks(umem, page_sz);
max_pgs_create_cmd =
(hwc->max_req_msg_size - sizeof(*create_req)) / sizeof(u64);
num_pages_to_handle =
min_t(size_t, num_pages_total, max_pgs_create_cmd);
create_req_msg_size =
struct_size(create_req, page_addr_list, num_pages_to_handle);
request_buf = kzalloc(hwc->max_req_msg_size, GFP_KERNEL);
if (!request_buf)
return -ENOMEM;
create_req = request_buf;
mana_gd_init_req_hdr(&create_req->hdr, GDMA_CREATE_DMA_REGION,
create_req_msg_size, sizeof(create_resp));
create_req->length = umem->length;
create_req->offset_in_page = umem->address & (page_sz - 1);
create_req->gdma_page_type = order_base_2(page_sz) - PAGE_SHIFT;
create_req->page_count = num_pages_total;
ibdev_dbg(&dev->ib_dev, "size_dma_region %lu num_pages_total %lu\n",
umem->length, num_pages_total);
ibdev_dbg(&dev->ib_dev, "page_sz %lu offset_in_page %u\n",
page_sz, create_req->offset_in_page);
ibdev_dbg(&dev->ib_dev, "num_pages_to_handle %lu, gdma_page_type %u",
num_pages_to_handle, create_req->gdma_page_type);
add_req = request_buf;
max_pgs_add_cmd =
(hwc->max_req_msg_size - sizeof(*add_req)) /
sizeof(u64);
page_addr_list = create_req->page_addr_list;
rdma_umem_for_each_dma_block(umem, &biter, page_sz) {
page_addr_list[tail++] = rdma_block_iter_dma_address(&biter);
if (tail >= num_pages_to_handle) {
u32 expected_s = 0;
if (num_pages_processed &&
num_pages_processed + num_pages_to_handle <
num_pages_total) {
/* Status indicating more pages are needed */
expected_s = GDMA_STATUS_MORE_ENTRIES;
}
if (!num_pages_processed) {
/* First message */
err = mana_ib_gd_first_dma_region(dev, gc,
create_req,
tail,
gdma_region);
if (err)
goto out;
page_addr_list = add_req->page_addr_list;
} else {
err = mana_ib_gd_add_dma_region(dev, gc,
add_req, tail,
expected_s);
if (err) {
tail = 0;
break;
}
}
num_pages_processed += tail;
/* Prepare to send ADD_PAGE requests */
num_pages_to_handle =
min_t(size_t,
num_pages_total - num_pages_processed,
max_pgs_add_cmd);
tail = 0;
}
}
if (tail) {
if (!num_pages_processed) {
err = mana_ib_gd_first_dma_region(dev, gc, create_req,
tail, gdma_region);
if (err)
goto out;
} else {
err = mana_ib_gd_add_dma_region(dev, gc, add_req,
tail, 0);
}
}
if (err)
mana_ib_gd_destroy_dma_region(dev, create_resp.dma_region_handle);out:
kfree(request_buf);
return err;
}Jason Gunthorpe
maintainerRe: [Patch v9 12/12] RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter
Quoted 5 lines · Long Li
page_addr_list = create_req->page_addr_list;
rdma_umem_for_each_dma_block(umem, &biter, page_sz) {
page_addr_list[tail++] = rdma_block_iter_dma_address(&biter);
if (tail >= num_pages_to_handle) {if (tail <= num_pages_to_handle)
continue
And remove a level of indentation
Quoted 44 lines
u32 expected_s = 0;
if (num_pages_processed &&
num_pages_processed + num_pages_to_handle <
num_pages_total) {
/* Status indicating more pages are needed */
expected_s = GDMA_STATUS_MORE_ENTRIES;
}
if (!num_pages_processed) {
/* First message */
err = mana_ib_gd_first_dma_region(dev, gc,
create_req,
tail,
gdma_region);
if (err)
goto out;
page_addr_list = add_req->page_addr_list;
} else {
err = mana_ib_gd_add_dma_region(dev, gc,
add_req, tail,
expected_s);
if (err) {
tail = 0;
break;
}
}
num_pages_processed += tail;
/* Prepare to send ADD_PAGE requests */
num_pages_to_handle =
min_t(size_t,
num_pages_total - num_pages_processed,
max_pgs_add_cmd);
tail = 0;
}
}
if (tail) {
if (!num_pages_processed) {
err = mana_ib_gd_first_dma_region(dev, gc, create_req,Quoted 8 lines
tail, gdma_region);
if (err)
goto out;
} else {
err = mana_ib_gd_add_dma_region(dev, gc, add_req,
tail, 0);
}
}Usually this can be folded above by having the first if not continue if the end of the list is reached.
Anyhow, this is much better
Thanks,
Jason
RE: [Patch v9 12/12] RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter
Quoted 66 lines · Long Li
Quoted 5 lines
page_addr_list = create_req->page_addr_list; rdma_umem_for_each_dma_block(umem, &biter, page_sz) { page_addr_list[tail++] = rdma_block_iter_dma_address(&biter); if (tail >= num_pages_to_handle) {if (tail <=
num_pages_to_handle)
continueAnd remove a level of indentation
Quoted 44 lines
u32 expected_s = 0; if (num_pages_processed && num_pages_processed + num_pages_to_handle < num_pages_total) { /* Status indicating more pages are needed */ expected_s = GDMA_STATUS_MORE_ENTRIES; } if (!num_pages_processed) { /* First message */ err = mana_ib_gd_first_dma_region(dev, gc, create_req, tail, gdma_region); if (err) goto out; page_addr_list = add_req->page_addr_list; } else { err = mana_ib_gd_add_dma_region(dev, gc, add_req, tail, expected_s); if (err) { tail = 0; break; } } num_pages_processed += tail; /* Prepare to send ADD_PAGE requests */ num_pages_to_handle = min_t(size_t, num_pages_total - num_pages_processed, max_pgs_add_cmd); tail = 0; } } if (tail) { if (!num_pages_processed) { err = mana_ib_gd_first_dma_region(dev, gc, create_req,Quoted 8 lines
tail, gdma_region); if (err) goto out; } else { err = mana_ib_gd_add_dma_region(dev, gc, add_req, tail, 0); } }
Quoted 7 lines
Usually this can be folded above by having the first if not continue if the end of the list is reached.
Anyhow, this is much better
Thanks,
Jason
Thank you, I'm changing to those in the next patch.
Long