RDMA/erdma: Support the RoCEv2 protocol
Subsystems (2)
v2 cover RDMA/erdma: Support the RoCEv2 protocol
Hi,
This patch series introduces support for the RoCEv2 protocol into the erdma driver. As the most prevalent RDMA protocol, RoCEv2 is widely used in the production environment. Given the extensive application of erdma across various scenarios in the Alibaba Cloud, there has arisen a requirement for erdma to support the RoCEv2 protocol. Therefore, we update both the erdma hardware and the erdma driver to accommodate the RoCEv2 protocol.
- #1 adds the device probe logic for the erdma RoCEv2 device.
- #2~#4 implement resource management interfaces required by the erdma RoCEv2 device, such as the GID table, PKey and address handle.
- #5~#6 implment the modify QP interface for the erdma RoCEv2 device and refactor the code of modify_qp interface to improve readability.
- #7 introduces the query_qp command to obtain several qp attributes from the hardware.
- #8 extends the procedure for create_qp, post_send and poll_cq interfaces to support UD QPs and UD WRs.
Thanks,
Boshi Yu
---
v2:
- Patch#1: remove the unnecessary check for device protocol.
- Patch#1: remove the erdma_device_init_iwarp() and erdma_device_init_rocev2() functions, as they are simple and used only once.
- Patch#4: move the function definitions of erdma_av_to_attr(), erdma_attr_to_av(), and erdma_set_av_cfg() to patch#5, where they are used for more than once.
- Patch#6: remove the declaration of deprecated erdma_modify_qp_rocev2().
- Patch#6: replace the 'reformat' with 'refactor' in the commit message for clarity.
v1:
Boshi Yu (8):
RDMA/erdma: Probe the erdma RoCEv2 device RDMA/erdma: Add GID table management interfaces RDMA/erdma: Add the erdma_query_pkey() interface RDMA/erdma: Add address handle implementation RDMA/erdma: Add erdma_modify_qp_rocev2() interface RDMA/erdma: Refactor the code of the modify_qp interface RDMA/erdma: Add the query_qp command to the cmdq RDMA/erdma: Support UD QPs and UD WRs
drivers/infiniband/hw/erdma/Kconfig | 2 +- drivers/infiniband/hw/erdma/erdma.h | 8 +- drivers/infiniband/hw/erdma/erdma_cm.c | 71 +-- drivers/infiniband/hw/erdma/erdma_cq.c | 65 +++ drivers/infiniband/hw/erdma/erdma_hw.h | 135 +++++- drivers/infiniband/hw/erdma/erdma_main.c | 44 +- drivers/infiniband/hw/erdma/erdma_qp.c | 299 +++++++++--- drivers/infiniband/hw/erdma/erdma_verbs.c | 524 +++++++++++++++++++--- drivers/infiniband/hw/erdma/erdma_verbs.h | 166 +++++-- 9 files changed, 1116 insertions(+), 198 deletions(-)
Signature · 1 line
2.43.5
PATCH v2 1/8 RDMA/erdma: Probe the erdma RoCEv2 device
Currently, the erdma driver supports both the iWARP and RoCEv2 protocols. The erdma driver reads the ERDMA_REGS_DEV_PROTO_REG register to identify the protocol used by the erdma device. Since each protocol requires different ib_device_ops, we introduce the erdma_device_ops_iwarp and erdma_device_ops_rocev2 for iWARP and RoCEv2 protocols, respectively.
--- drivers/infiniband/hw/erdma/Kconfig | 2 +- drivers/infiniband/hw/erdma/erdma.h | 3 +- drivers/infiniband/hw/erdma/erdma_hw.h | 7 +++++ drivers/infiniband/hw/erdma/erdma_main.c | 34 +++++++++++++++++------ drivers/infiniband/hw/erdma/erdma_verbs.c | 16 ++++++++++- drivers/infiniband/hw/erdma/erdma_verbs.h | 12 ++++++++ 6 files changed, 62 insertions(+), 12 deletions(-)
drivers/infiniband/hw/erdma/Kconfig +1 -1
@@ -5,7 +5,7 @@ config INFINIBAND_ERDMA depends on INFINIBAND_ADDR_TRANS depends on INFINIBAND_USER_ACCESS help - This is a RDMA/iWarp driver for Alibaba Elastic RDMA Adapter(ERDMA), + This is a RDMA driver for Alibaba Elastic RDMA Adapter(ERDMA), which supports RDMA features in Alibaba cloud environment. To compile this driver as module, choose M here. The module will be
drivers/infiniband/hw/erdma/erdma.h +2 -1
@@ -16,7 +16,7 @@#include "erdma_hw.h" #define DRV_MODULE_NAME "erdma" -#define ERDMA_NODE_DESC "Elastic RDMA(iWARP) stack" +#define ERDMA_NODE_DESC "Elastic RDMA Adapter stack"struct erdma_eq {void *qbuf;
@@ -215,6 +215,7 @@ struct erdma_dev {struct dma_pool *db_pool;struct dma_pool *resp_pool; +enum erdma_proto_type proto; };static inline void *get_queue_entry (void *qbuf,u32 idx,u32 depth,u32 shift)
drivers/infiniband/hw/erdma/erdma_hw.h +7 -0
@@ -21,8 +21,15 @@#define ERDMA_NUM_MSIX_VEC 32U#define ERDMA_MSIX_VECTOR_CMDQ 0 +/* erdma device protocol type */ +enum erdma_proto_type { +ERDMA_PROTO_IWARP =0 , +ERDMA_PROTO_ROCEV2 =1 , +}; +/* PCIe Bar0 Registers. */ #define ERDMA_REGS_VERSION_REG 0x0 +#define ERDMA_REGS_DEV_PROTO_REG 0xC#define ERDMA_REGS_DEV_CTRL_REG 0x10#define ERDMA_REGS_DEV_ST_REG 0x14#define ERDMA_REGS_NETDEV_MAC_L_REG 0x18
drivers/infiniband/hw/erdma/erdma_main.c +25 -9
@@ -172,6 +172,8 @@ static int erdma_device_init(struct erdma_dev *dev, struct pci_dev *pdev) {int ret; + dev->proto =erdma_reg_read32 (dev,ERDMA_REGS_DEV_PROTO_REG ); + dev->resp_pool =dma_pool_create ("erdma_resp_pool" , &pdev->dev,ERDMA_HW_RESP_SIZE ,ERDMA_HW_RESP_SIZE ,0 );
@@ -474,6 +476,21 @@ static void erdma_res_cb_free(struct erdma_dev *dev)bitmap_free (dev->res_cb[i].bitmap); } +static const struct ib_device_ops erdma_device_ops_rocev2 = { + .get_link_layer = erdma_get_link_layer, +}; + +static const struct ib_device_ops erdma_device_ops_iwarp = { + .iw_accept = erdma_accept, + .iw_add_ref = erdma_qp_get_ref, + .iw_connect = erdma_connect, + .iw_create_listen = erdma_create_listen, + .iw_destroy_listen = erdma_destroy_listen, + .iw_get_qp = erdma_get_ibqp, + .iw_reject = erdma_reject, + .iw_rem_ref = erdma_qp_put_ref, +}; +static const struct ib_device_ops erdma_device_ops = { .owner =THIS_MODULE , .driver_id =RDMA_DRIVER_ERDMA ,
@@ -494,14 +511,6 @@ static const struct ib_device_ops erdma_device_ops = { .get_dma_mr = erdma_get_dma_mr, .get_hw_stats = erdma_get_hw_stats, .get_port_immutable = erdma_get_port_immutable, - .iw_accept = erdma_accept, - .iw_add_ref = erdma_qp_get_ref, - .iw_connect = erdma_connect, - .iw_create_listen = erdma_create_listen, - .iw_destroy_listen = erdma_destroy_listen, - .iw_get_qp = erdma_get_ibqp, - .iw_reject = erdma_reject, - .iw_rem_ref = erdma_qp_put_ref, .map_mr_sg = erdma_map_mr_sg, .mmap = erdma_mmap, .mmap_free = erdma_mmap_free,
@@ -537,7 +546,14 @@ static int erdma_ib_device_add(struct pci_dev *pdev)if (ret)return ret; - ibdev->node_type =RDMA_NODE_RNIC ; +if (erdma_device_iwarp (dev)) { + ibdev->node_type =RDMA_NODE_RNIC ; +ib_set_device_ops (ibdev, &erdma_device_ops_iwarp); + }else { + ibdev->node_type =RDMA_NODE_IB_CA ; +ib_set_device_ops (ibdev, &erdma_device_ops_rocev2); + } +memcpy (ibdev->node_desc,ERDMA_NODE_DESC ,sizeof (ERDMA_NODE_DESC ));/*
drivers/infiniband/hw/erdma/erdma_verbs.c +15 -1
@@ -395,8 +395,17 @@ int erdma_query_port(struct ib_device *ibdev, u32 port,int erdma_get_port_immutable (struct ib_device *ibdev,u32 port,struct ib_port_immutable *port_immutable) { +struct erdma_dev *dev =to_edev (ibdev); + +if (erdma_device_iwarp (dev)) { + port_immutable->core_cap_flags =RDMA_CORE_PORT_IWARP ; + }else { + port_immutable->core_cap_flags = +RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP ; + port_immutable->max_mad_size =IB_MGMT_MAD_SIZE ; + } + port_immutable->gid_tbl_len =1 ; - port_immutable->core_cap_flags =RDMA_CORE_PORT_IWARP ;return 0 ; }
@@ -1839,3 +1848,8 @@ int erdma_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats,return stats->num_counters; } + +enum rdma_link_layer erdma_get_link_layer (struct ib_device *ibdev,u32 port_num) +{ +return IB_LINK_LAYER_ETHERNET ; +}
drivers/infiniband/hw/erdma/erdma_verbs.h +12 -0
@@ -291,6 +291,16 @@ int erdma_modify_qp_internal(struct erdma_qp *qp, struct erdma_qp_attrs *attrs,void erdma_qp_llp_close (struct erdma_qp *qp);void erdma_qp_cm_drop (struct erdma_qp *qp); +static inline bool erdma_device_iwarp (struct erdma_dev *dev) +{ +return dev->proto ==ERDMA_PROTO_IWARP ; +} + +static inline bool erdma_device_rocev2 (struct erdma_dev *dev) +{ +return dev->proto ==ERDMA_PROTO_ROCEV2 ; +} +static inline struct erdma_ucontext *to_ectx (struct ib_ucontext *ibctx) { return container_of(ibctx, struct erdma_ucontext, ibucontext);
@@ -370,5 +380,7 @@ struct rdma_hw_stats *erdma_alloc_hw_port_stats(struct ib_device *device, u32 port_num);int erdma_get_hw_stats (struct ib_device *ibdev,struct rdma_hw_stats *stats,u32 port,int index); +enum rdma_link_layer erdma_get_link_layer (struct ib_device *ibdev, +u32 port_num);#endif
Signature · 1 line
2.43.5
PATCH v2 2/8 RDMA/erdma: Add GID table management interfaces
In reply to Boshi YuThe erdma_add_gid() interface inserts a GID entry at the specified index. The erdma_del_gid() interface deletes the GID entry at the specified index. Additionally, programs can invoke the erdma_query_port() and erdma_get_port_immutable() interfaces to query the GID table length.
--- drivers/infiniband/hw/erdma/erdma.h | 1 + drivers/infiniband/hw/erdma/erdma_hw.h | 28 +++++++++++- drivers/infiniband/hw/erdma/erdma_main.c | 3 ++ drivers/infiniband/hw/erdma/erdma_verbs.c | 56 +++++++++++++++++++++-- drivers/infiniband/hw/erdma/erdma_verbs.h | 12 +++++ 5 files changed, 96 insertions(+), 4 deletions(-)
drivers/infiniband/hw/erdma/erdma.h +1 -0
@@ -148,6 +148,7 @@ struct erdma_devattr {u32 max_mr;u32 max_pd;u32 max_mw; +u32 max_gid;u32 local_dma_key; };
drivers/infiniband/hw/erdma/erdma_hw.h +27 -1
@@ -21,6 +21,9 @@#define ERDMA_NUM_MSIX_VEC 32U#define ERDMA_MSIX_VECTOR_CMDQ 0 +/* RoCEv2 related */ +#define ERDMA_ROCEV2_GID_SIZE 16 +/* erdma device protocol type */ enum erdma_proto_type {ERDMA_PROTO_IWARP =0 ,
@@ -143,7 +146,8 @@ enum CMDQ_RDMA_OPCODE {CMDQ_OPCODE_DESTROY_CQ =5 ,CMDQ_OPCODE_REFLUSH =6 ,CMDQ_OPCODE_REG_MR =8 , -CMDQ_OPCODE_DEREG_MR =9 +CMDQ_OPCODE_DEREG_MR =9 , + CMDQ_OPCODE_SET_GID = 14, }; enum CMDQ_COMMON_OPCODE {
@@ -401,7 +405,29 @@ struct erdma_cmdq_query_stats_resp {u64 rx_pps_meter_drop_packets_cnt; }; +enum erdma_network_type { +ERDMA_NETWORK_TYPE_IPV4 =0 , +ERDMA_NETWORK_TYPE_IPV6 =1 , +}; + +enum erdma_set_gid_op { +ERDMA_SET_GID_OP_ADD =0 , +ERDMA_SET_GID_OP_DEL =1 , +}; + +/* set gid cfg */ +#define ERDMA_CMD_SET_GID_SGID_IDX_MASK GENMASK(15, 0) +#define ERDMA_CMD_SET_GID_NTYPE_MASK BIT(16) +#define ERDMA_CMD_SET_GID_OP_MASK BIT(31) + +struct erdma_cmdq_set_gid_req { +u64 hdr; +u32 cfg; +u8 gid[ERDMA_ROCEV2_GID_SIZE ]; +}; +/* cap qword 0 definition */ +#define ERDMA_CMD_DEV_CAP_MAX_GID_MASK GENMASK_ULL(51, 48)#define ERDMA_CMD_DEV_CAP_MAX_CQE_MASK GENMASK_ULL(47, 40)#define ERDMA_CMD_DEV_CAP_FLAGS_MASK GENMASK_ULL(31, 24)#define ERDMA_CMD_DEV_CAP_MAX_RECV_WR_MASK GENMASK_ULL(23, 16)
drivers/infiniband/hw/erdma/erdma_main.c +3 -0
@@ -400,6 +400,7 @@ static int erdma_dev_attrs_init(struct erdma_dev *dev) dev->attrs.max_mr_size =1ULL <<ERDMA_GET_CAP (MAX_MR_SIZE , cap0); dev->attrs.max_mw =1 <<ERDMA_GET_CAP (MAX_MW , cap1); dev->attrs.max_recv_wr =1 <<ERDMA_GET_CAP (MAX_RECV_WR , cap0); + dev->attrs.max_gid =1 <<ERDMA_GET_CAP (MAX_GID , cap0); dev->attrs.local_dma_key =ERDMA_GET_CAP (DMA_LOCAL_KEY , cap1); dev->attrs.cc =ERDMA_GET_CAP (DEFAULT_CC , cap1); dev->attrs.max_qp =ERDMA_NQP_PER_QBLOCK *ERDMA_GET_CAP (QBLOCK , cap1);
@@ -478,6 +479,8 @@ static void erdma_res_cb_free(struct erdma_dev *dev)static const struct ib_device_ops erdma_device_ops_rocev2 = { .get_link_layer = erdma_get_link_layer, + .add_gid = erdma_add_gid, + .del_gid = erdma_del_gid, };static const struct ib_device_ops erdma_device_ops_iwarp = {
drivers/infiniband/hw/erdma/erdma_verbs.c +53 -3
77 lines not on this page · raw message
drivers/infiniband/hw/erdma/erdma_verbs.h +12 -0
@@ -326,6 +326,16 @@ static inline struct erdma_cq *to_ecq(struct ib_cq *ibcq) return container_of(ibcq, struct erdma_cq, ibcq); } +static inline int erdma_check_gid_attr (const struct ib_gid_attr *attr) +{ +u8 ntype =rdma_gid_attr_network_type (attr); + +if (ntype !=RDMA_NETWORK_IPV4 && ntype !=RDMA_NETWORK_IPV6 ) +return -EINVAL ; + +return 0 ; +} +static inline struct erdma_user_mmap_entry *to_emmap (struct rdma_user_mmap_entry *ibmmap) {
@@ -382,5 +392,7 @@ int erdma_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats, u32 port, int index);enum rdma_link_layer erdma_get_link_layer (struct ib_device *ibdev,u32 port_num); +int erdma_add_gid (const struct ib_gid_attr *attr,void **context); +int erdma_del_gid (const struct ib_gid_attr *attr,void **context);#endif
Signature · 1 line
2.43.5
PATCH v2 3/8 RDMA/erdma: Add the erdma_query_pkey() interface
In reply to Boshi YuThe erdma_query_pkey() interface queries the PKey at the specified index. Currently, erdma supports only one partition and returns the default PKey for each query. Besides, the correct length of the PKey table can be obtained by calling the erdma_query_port() and erdma_get_port_immutable() interfaces.
--- drivers/infiniband/hw/erdma/erdma_hw.h | 2 ++ drivers/infiniband/hw/erdma/erdma_main.c | 1 + drivers/infiniband/hw/erdma/erdma_verbs.c | 14 ++++++++++++++ drivers/infiniband/hw/erdma/erdma_verbs.h | 1 + 4 files changed, 18 insertions(+)
drivers/infiniband/hw/erdma/erdma_hw.h +2 -0
@@ -23,6 +23,8 @@/* RoCEv2 related */ #define ERDMA_ROCEV2_GID_SIZE 16 +#define ERDMA_MAX_PKEYS 1 +#define ERDMA_DEFAULT_PKEY 0xFFFF/* erdma device protocol type */ enum erdma_proto_type {
drivers/infiniband/hw/erdma/erdma_main.c +1 -0
@@ -481,6 +481,7 @@ static const struct ib_device_ops erdma_device_ops_rocev2 = { .get_link_layer = erdma_get_link_layer, .add_gid = erdma_add_gid, .del_gid = erdma_del_gid, + .query_pkey = erdma_query_pkey, };static const struct ib_device_ops erdma_device_ops_iwarp = {
drivers/infiniband/hw/erdma/erdma_verbs.c +14 -0
@@ -336,6 +336,9 @@ int erdma_query_device(struct ib_device *ibdev, struct ib_device_attr *attr, attr->max_fast_reg_page_list_len =ERDMA_MAX_FRMR_PA ; attr->page_size_cap =ERDMA_PAGE_SIZE_SUPPORT ; +if (erdma_device_rocev2 (dev)) + attr->max_pkeys =ERDMA_MAX_PKEYS ; +if (dev->attrs.cap_flags &ERDMA_DEV_CAP_FLAGS_ATOMIC ) attr->atomic_cap =IB_ATOMIC_GLOB ;
@@ -372,6 +375,7 @@ int erdma_query_port(struct ib_device *ibdev, u32 port, }else { attr->gid_tbl_len = dev->attrs.max_gid; attr->ip_gids = true; + attr->pkey_tbl_len =ERDMA_MAX_PKEYS ; } attr->port_cap_flags =IB_PORT_CM_SUP |IB_PORT_DEVICE_MGMT_SUP ;
@@ -411,6 +415,7 @@ int erdma_get_port_immutable(struct ib_device *ibdev, u32 port,RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP ; port_immutable->max_mad_size =IB_MGMT_MAD_SIZE ; port_immutable->gid_tbl_len = dev->attrs.max_gid; + port_immutable->pkey_tbl_len =ERDMA_MAX_PKEYS ; }return 0 ;
@@ -1903,3 +1908,12 @@ int erdma_del_gid(const struct ib_gid_attr *attr, void **context)return erdma_set_gid (to_edev (attr->device),ERDMA_SET_GID_OP_DEL , attr->index,NULL ); } + +int erdma_query_pkey (struct ib_device *ibdev,u32 port,u16 index,u16 *pkey) +{ +if (index >=ERDMA_MAX_PKEYS ) +return -EINVAL ; + + *pkey =ERDMA_DEFAULT_PKEY ; +return 0 ; +}
drivers/infiniband/hw/erdma/erdma_verbs.h +1 -0
@@ -394,5 +394,6 @@ enum rdma_link_layer erdma_get_link_layer(struct ib_device *ibdev, u32 port_num);int erdma_add_gid (const struct ib_gid_attr *attr,void **context);int erdma_del_gid (const struct ib_gid_attr *attr,void **context); +int erdma_query_pkey (struct ib_device *ibdev,u32 port,u16 index,u16 *pkey);#endif
Signature · 1 line
2.43.5
PATCH v2 4/8 RDMA/erdma: Add address handle implementation
In reply to Boshi YuThe address handle contains the necessary information to transmit messages to a remote peer in the RoCEv2 protocol. This commit implements the erdma_create_ah(), erdma_destroy_ah(), and erdma_query_ah() interfaces, which are used to create, destroy, and query an address handle, respectively.
--- drivers/infiniband/hw/erdma/erdma.h | 4 +- drivers/infiniband/hw/erdma/erdma_hw.h | 34 +++++++ drivers/infiniband/hw/erdma/erdma_main.c | 4 + drivers/infiniband/hw/erdma/erdma_verbs.c | 114 +++++++++++++++++++++- drivers/infiniband/hw/erdma/erdma_verbs.h | 28 ++++++ 5 files changed, 182 insertions(+), 2 deletions(-)
drivers/infiniband/hw/erdma/erdma.h +3 -1
@@ -149,6 +149,7 @@ struct erdma_devattr {u32 max_pd;u32 max_mw;u32 max_gid; +u32 max_ah;u32 local_dma_key; };
@@ -178,7 +179,8 @@ struct erdma_resource_cb {enum {ERDMA_RES_TYPE_PD =0 ,ERDMA_RES_TYPE_STAG_IDX =1 , -ERDMA_RES_CNT =2 , +ERDMA_RES_TYPE_AH =2 , +ERDMA_RES_CNT =3 , };struct erdma_dev {
drivers/infiniband/hw/erdma/erdma_hw.h +34 -0
@@ -9,6 +9,7 @@#include <linux/kernel.h> #include <linux/types.h> +#include <linux/if_ether.h> /* PCIe device related definition. */ #define ERDMA_PCI_WIDTH 64
@@ -150,6 +151,8 @@ enum CMDQ_RDMA_OPCODE {CMDQ_OPCODE_REG_MR =8 ,CMDQ_OPCODE_DEREG_MR =9 ,CMDQ_OPCODE_SET_GID =14 , +CMDQ_OPCODE_CREATE_AH =15 , + CMDQ_OPCODE_DESTROY_AH = 16, }; enum CMDQ_COMMON_OPCODE {
@@ -297,6 +300,36 @@ struct erdma_cmdq_dereg_mr_req {u32 cfg; }; +/* create_av cfg0 */ +#define ERDMA_CMD_CREATE_AV_FL_MASK GENMASK(19, 0) +#define ERDMA_CMD_CREATE_AV_NTYPE_MASK BIT(20) + +struct erdma_av_cfg { +u32 cfg0; +u8 traffic_class; +u8 hop_limit; +u8 sl; +u8 rsvd; +u16 udp_sport; +u16 sgid_index; +u8 dmac[ETH_ALEN ]; +u8 padding[2 ]; +u8 dgid[ERDMA_ROCEV2_GID_SIZE ]; +}; + +struct erdma_cmdq_create_ah_req { +u64 hdr; +u32 pdn; +u32 ahn; +struct erdma_av_cfg av_cfg; +}; + +struct erdma_cmdq_destroy_ah_req { +u64 hdr; +u32 pdn; +u32 ahn; +}; +/* modify qp cfg */ #define ERDMA_CMD_MODIFY_QP_STATE_MASK GENMASK(31, 24)#define ERDMA_CMD_MODIFY_QP_CC_MASK GENMASK(23, 20)
@@ -433,6 +466,7 @@ struct erdma_cmdq_set_gid_req {#define ERDMA_CMD_DEV_CAP_MAX_CQE_MASK GENMASK_ULL(47, 40)#define ERDMA_CMD_DEV_CAP_FLAGS_MASK GENMASK_ULL(31, 24)#define ERDMA_CMD_DEV_CAP_MAX_RECV_WR_MASK GENMASK_ULL(23, 16) +#define ERDMA_CMD_DEV_CAP_MAX_AH_MASK GENMASK_ULL(15, 8)#define ERDMA_CMD_DEV_CAP_MAX_MR_SIZE_MASK GENMASK_ULL(7, 0)/* cap qword 1 definition */
drivers/infiniband/hw/erdma/erdma_main.c +4 -0
@@ -401,6 +401,7 @@ static int erdma_dev_attrs_init(struct erdma_dev *dev) dev->attrs.max_mw =1 <<ERDMA_GET_CAP (MAX_MW , cap1); dev->attrs.max_recv_wr =1 <<ERDMA_GET_CAP (MAX_RECV_WR , cap0); dev->attrs.max_gid =1 <<ERDMA_GET_CAP (MAX_GID , cap0); + dev->attrs.max_ah =1 <<ERDMA_GET_CAP (MAX_AH , cap0); dev->attrs.local_dma_key =ERDMA_GET_CAP (DMA_LOCAL_KEY , cap1); dev->attrs.cc =ERDMA_GET_CAP (DEFAULT_CC , cap1); dev->attrs.max_qp =ERDMA_NQP_PER_QBLOCK *ERDMA_GET_CAP (QBLOCK , cap1);
@@ -418,6 +419,7 @@ static int erdma_dev_attrs_init(struct erdma_dev *dev) dev->res_cb[ERDMA_RES_TYPE_PD ].max_cap =ERDMA_MAX_PD ; dev->res_cb[ERDMA_RES_TYPE_STAG_IDX ].max_cap = dev->attrs.max_mr; + dev->res_cb[ERDMA_RES_TYPE_AH ].max_cap = dev->attrs.max_ah;erdma_cmdq_build_reqhdr (&req_hdr,CMDQ_SUBMOD_COMMON ,CMDQ_OPCODE_QUERY_FW_INFO );
@@ -482,6 +484,8 @@ static const struct ib_device_ops erdma_device_ops_rocev2 = { .add_gid = erdma_add_gid, .del_gid = erdma_del_gid, .query_pkey = erdma_query_pkey, + .create_ah = erdma_create_ah, + .destroy_ah = erdma_destroy_ah, };static const struct ib_device_ops erdma_device_ops_iwarp = {
drivers/infiniband/hw/erdma/erdma_verbs.c +113 -1
124 lines not on this page · raw message
drivers/infiniband/hw/erdma/erdma_verbs.h +28 -0
@@ -136,6 +136,25 @@ struct erdma_user_dbrecords_page {int refcnt; }; +struct erdma_av { +u8 port; +u8 hop_limit; +u8 traffic_class; +u8 sl; +u8 sgid_index; +u16 udp_sport; +u32 flow_label; +u8 dmac[ETH_ALEN ]; +u8 dgid[ERDMA_ROCEV2_GID_SIZE ]; +enum erdma_network_type ntype; +}; + +struct erdma_ah { +struct ib_ah ibah; +struct erdma_av av; +u32 ahn; +}; +struct erdma_uqp {struct erdma_mem sq_mem;struct erdma_mem rq_mem;
@@ -326,6 +345,11 @@ static inline struct erdma_cq *to_ecq(struct ib_cq *ibcq) return container_of(ibcq, struct erdma_cq, ibcq); } +static inline struct erdma_ah *to_eah (struct ib_ah *ibah) +{ + return container_of(ibah, struct erdma_ah, ibah); +} +static inline int erdma_check_gid_attr (const struct ib_gid_attr *attr) {u8 ntype =rdma_gid_attr_network_type (attr);
@@ -395,5 +419,9 @@ enum rdma_link_layer erdma_get_link_layer(struct ib_device *ibdev,int erdma_add_gid (const struct ib_gid_attr *attr,void **context);int erdma_del_gid (const struct ib_gid_attr *attr,void **context);int erdma_query_pkey (struct ib_device *ibdev,u32 port,u16 index,u16 *pkey); +int erdma_create_ah (struct ib_ah *ibah,struct rdma_ah_init_attr *init_attr, +struct ib_udata *udata); +int erdma_destroy_ah (struct ib_ah *ibah,u32 flags); +int erdma_query_ah (struct ib_ah *ibah,struct rdma_ah_attr *ah_attr);#endif
Signature · 1 line
2.43.5
PATCH v2 5/8 RDMA/erdma: Add erdma_modify_qp_rocev2() interface
In reply to Boshi YuThe QP state machines in the RoCEv2 and iWARP protocols are different. To handle these differences for the erdma RoCEv2 device, we provide the erdma_modify_qp_rocev2() interface, which transitions the QP state and modifies QP attributes accordingly.
--- drivers/infiniband/hw/erdma/erdma_cq.c | 45 +++++ drivers/infiniband/hw/erdma/erdma_hw.h | 14 ++ drivers/infiniband/hw/erdma/erdma_main.c | 3 +- drivers/infiniband/hw/erdma/erdma_qp.c | 92 +++++++++ drivers/infiniband/hw/erdma/erdma_verbs.c | 221 +++++++++++++++++++--- drivers/infiniband/hw/erdma/erdma_verbs.h | 44 +++++ 6 files changed, 388 insertions(+), 31 deletions(-)
drivers/infiniband/hw/erdma/erdma_cq.c +45 -0
@@ -201,3 +201,48 @@ int erdma_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)return npolled; } + +void erdma_remove_cqes_of_qp (struct ib_cq *ibcq,u32 qpn) +{ +struct erdma_cq *cq =to_ecq (ibcq); +struct erdma_cqe *cqe, *dst_cqe; +u32 prev_cq_ci, cur_cq_ci; +u32 ncqe =0 , nqp_cqe =0 ; +unsigned long flags; +u8 owner; + +spin_lock_irqsave (&cq->kern_cq.lock, flags); + + prev_cq_ci = cq->kern_cq.ci; + +while (ncqe < cq->depth && (cqe =get_next_valid_cqe (cq)) !=NULL ) { + ++cq->kern_cq.ci; + ++ncqe; + } + +while (ncqe >0 ) { + cur_cq_ci = prev_cq_ci + ncqe -1 ; + cqe =get_queue_entry (cq->kern_cq.qbuf, cur_cq_ci, cq->depth, +CQE_SHIFT ); + +if (be32_to_cpu (cqe->qpn) == qpn) { + ++nqp_cqe; + }else if (nqp_cqe) { + dst_cqe =get_queue_entry (cq->kern_cq.qbuf, + cur_cq_ci + nqp_cqe, + cq->depth,CQE_SHIFT ); + owner =FIELD_GET (ERDMA_CQE_HDR_OWNER_MASK , +be32_to_cpu (dst_cqe->hdr)); + cqe->hdr =cpu_to_be32 ( + (be32_to_cpu (cqe->hdr) & + ~ERDMA_CQE_HDR_OWNER_MASK ) | +FIELD_PREP (ERDMA_CQE_HDR_OWNER_MASK , owner)); +memcpy (dst_cqe, cqe,sizeof (*cqe)); + } + + --ncqe; + } + + cq->kern_cq.ci = prev_cq_ci + nqp_cqe; +spin_unlock_irqrestore (&cq->kern_cq.lock, flags); +}
drivers/infiniband/hw/erdma/erdma_hw.h +14 -0
@@ -347,6 +347,20 @@ struct erdma_cmdq_modify_qp_req {u32 recv_nxt; }; +/* modify qp cfg1 for roce device */ +#define ERDMA_CMD_MODIFY_QP_DQPN_MASK GENMASK(19, 0) + +struct erdma_cmdq_mod_qp_req_rocev2 { +u64 hdr; +u32 cfg0; +u32 cfg1; +u32 attr_mask; +u32 qkey; +u32 rq_psn; +u32 sq_psn; +struct erdma_av_cfg av_cfg; +}; +/* create qp cfg0 */ #define ERDMA_CMD_CREATE_QP_SQ_DEPTH_MASK GENMASK(31, 20)#define ERDMA_CMD_CREATE_QP_QPN_MASK GENMASK(19, 0)
drivers/infiniband/hw/erdma/erdma_main.c +2 -1
@@ -486,6 +486,7 @@ static const struct ib_device_ops erdma_device_ops_rocev2 = { .query_pkey = erdma_query_pkey, .create_ah = erdma_create_ah, .destroy_ah = erdma_destroy_ah, + .modify_qp = erdma_modify_qp_rocev2, };static const struct ib_device_ops erdma_device_ops_iwarp = {
@@ -497,6 +498,7 @@ static const struct ib_device_ops erdma_device_ops_iwarp = { .iw_get_qp = erdma_get_ibqp, .iw_reject = erdma_reject, .iw_rem_ref = erdma_qp_put_ref, + .modify_qp = erdma_modify_qp, };static const struct ib_device_ops erdma_device_ops = {
@@ -522,7 +524,6 @@ static const struct ib_device_ops erdma_device_ops = { .map_mr_sg = erdma_map_mr_sg, .mmap = erdma_mmap, .mmap_free = erdma_mmap_free, - .modify_qp = erdma_modify_qp, .post_recv = erdma_post_recv, .post_send = erdma_post_send, .poll_cq = erdma_poll_cq,
drivers/infiniband/hw/erdma/erdma_qp.c +92 -0
98 lines not on this page · raw message
drivers/infiniband/hw/erdma/erdma_verbs.c +191 -30
274 lines not on this page · raw message
drivers/infiniband/hw/erdma/erdma_verbs.h +44 -0
84 lines not on this page · raw message
Signature · 1 line
2.43.5
PATCH v2 6/8 RDMA/erdma: Refactor the code of the modify_qp interface
In reply to Boshi YuThe procedure for modifying QP is similar for both the iWARP and RoCEv2 protocols. Therefore, we unify the code and provide the erdma_modify_qp() interface for both protocols.
--- drivers/infiniband/hw/erdma/erdma_cm.c | 71 ++++---- drivers/infiniband/hw/erdma/erdma_hw.h | 1 + drivers/infiniband/hw/erdma/erdma_main.c | 3 +- drivers/infiniband/hw/erdma/erdma_qp.c | 136 ++++++++------ drivers/infiniband/hw/erdma/erdma_verbs.c | 210 +++++++++++----------- drivers/infiniband/hw/erdma/erdma_verbs.h | 75 +++++--- 6 files changed, 276 insertions(+), 220 deletions(-)
drivers/infiniband/hw/erdma/erdma_cm.c +39 -32
@@ -567,7 +567,8 @@ static int erdma_proc_mpareq(struct erdma_cep *cep)static int erdma_proc_mpareply (struct erdma_cep *cep) { -struct erdma_qp_attrs qp_attrs; +enum erdma_qpa_mask_iwarp to_modify_attrs =0 ; +struct erdma_mod_qp_params_iwarp params;struct erdma_qp *qp = cep->qp;struct mpa_rr *rep;int ret;
@@ -597,26 +598,29 @@ static int erdma_proc_mpareply(struct erdma_cep *cep)return -EINVAL ; } -memset (&qp_attrs,0 ,sizeof (qp_attrs)); - qp_attrs.irq_size = cep->ird; - qp_attrs.orq_size = cep->ord; - qp_attrs.state =ERDMA_QP_STATE_RTS ; +memset (¶ms,0 ,sizeof (params)); + params.state =ERDMA_QPS_IWARP_RTS ; + params.irq_size = cep->ird; + params.orq_size = cep->ord;down_write (&qp->state_lock); -if (qp->attrs.state >ERDMA_QP_STATE_RTR ) { +if (qp->attrs.iwarp.state >ERDMA_QPS_IWARP_RTR ) { ret = -EINVAL ;up_write (&qp->state_lock); goto out_err; } - qp->attrs.qp_type =ERDMA_QP_ACTIVE ; -if (__mpa_ext_cc (cep->mpa.ext_data.bits) != qp->attrs.cc) - qp->attrs.cc =COMPROMISE_CC ; + to_modify_attrs =ERDMA_QPA_IWARP_STATE |ERDMA_QPA_IWARP_LLP_HANDLE | +ERDMA_QPA_IWARP_MPA |ERDMA_QPA_IWARP_IRD | +ERDMA_QPA_IWARP_ORD ; - ret =erdma_modify_qp_internal (qp, &qp_attrs, -ERDMA_QP_ATTR_STATE | -ERDMA_QP_ATTR_LLP_HANDLE | -ERDMA_QP_ATTR_MPA ); + params.qp_type =ERDMA_QP_ACTIVE ; +if (__mpa_ext_cc (cep->mpa.ext_data.bits) != qp->attrs.cc) { + to_modify_attrs |=ERDMA_QPA_IWARP_CC ; + params.cc =COMPROMISE_CC ; + } + + ret =erdma_modify_qp_state_iwarp (qp, ¶ms, to_modify_attrs);up_write (&qp->state_lock);
@@ -722,7 +726,7 @@ static int erdma_newconn_connected(struct erdma_cep *cep)__mpa_rr_set_revision (&cep->mpa.hdr.params.bits,MPA_REVISION_EXT_1 );memcpy (cep->mpa.hdr.key,MPA_KEY_REQ ,MPA_KEY_SIZE ); - cep->mpa.ext_data.cookie =cpu_to_be32 (cep->qp->attrs.cookie); + cep->mpa.ext_data.cookie =cpu_to_be32 (cep->qp->attrs.iwarp.cookie);__mpa_ext_set_cc (&cep->mpa.ext_data.bits, cep->qp->attrs.cc); ret =erdma_send_mpareqrep (cep, cep->private_data, cep->pd_len);
@@ -1126,10 +1130,11 @@ int erdma_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)int erdma_accept (struct iw_cm_id *id,struct iw_cm_conn_param *params) { -struct erdma_dev *dev =to_edev (id->device);struct erdma_cep *cep = (struct erdma_cep *)id->provider_data; +struct erdma_mod_qp_params_iwarp mod_qp_params; +enum erdma_qpa_mask_iwarp to_modify_attrs =0 ; +struct erdma_dev *dev =to_edev (id->device);struct erdma_qp *qp; -struct erdma_qp_attrs qp_attrs;int ret;erdma_cep_set_inuse (cep);
@@ -1156,7 +1161,7 @@ int erdma_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)erdma_qp_get (qp);down_write (&qp->state_lock); -if (qp->attrs.state >ERDMA_QP_STATE_RTR ) { +if (qp->attrs.iwarp.state >ERDMA_QPS_IWARP_RTR ) { ret = -EINVAL ;up_write (&qp->state_lock); goto error;
@@ -1181,11 +1186,11 @@ int erdma_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) cep->cm_id = id; id->add_ref (id); -memset (&qp_attrs,0 ,sizeof (qp_attrs)); - qp_attrs.orq_size = params->ord; - qp_attrs.irq_size = params->ird; +memset (&mod_qp_params,0 ,sizeof (mod_qp_params)); - qp_attrs.state =ERDMA_QP_STATE_RTS ; + mod_qp_params.irq_size = params->ird; + mod_qp_params.orq_size = params->ord; + mod_qp_params.state =ERDMA_QPS_IWARP_RTS ;/* Associate QP with CEP */ erdma_cep_get (cep);
@@ -1194,19 +1199,21 @@ int erdma_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) cep->state =ERDMA_EPSTATE_RDMA_MODE ; - qp->attrs.qp_type =ERDMA_QP_PASSIVE ; - qp->attrs.pd_len = params->private_data_len; + mod_qp_params.qp_type =ERDMA_QP_PASSIVE ; + mod_qp_params.pd_len = params->private_data_len; -if (qp->attrs.cc !=__mpa_ext_cc (cep->mpa.ext_data.bits)) - qp->attrs.cc =COMPROMISE_CC ; + to_modify_attrs =ERDMA_QPA_IWARP_STATE |ERDMA_QPA_IWARP_ORD | +ERDMA_QPA_IWARP_LLP_HANDLE |ERDMA_QPA_IWARP_IRD | +ERDMA_QPA_IWARP_MPA ; + +if (qp->attrs.cc !=__mpa_ext_cc (cep->mpa.ext_data.bits)) { + to_modify_attrs |=ERDMA_QPA_IWARP_CC ; + mod_qp_params.cc =COMPROMISE_CC ; + }/* move to rts */ - ret =erdma_modify_qp_internal (qp, &qp_attrs, -ERDMA_QP_ATTR_STATE | -ERDMA_QP_ATTR_ORD | -ERDMA_QP_ATTR_LLP_HANDLE | -ERDMA_QP_ATTR_IRD | -ERDMA_QP_ATTR_MPA ); + ret =erdma_modify_qp_state_iwarp (qp, &mod_qp_params, to_modify_attrs); +up_write (&qp->state_lock);if (ret)
@@ -1214,7 +1221,7 @@ int erdma_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params) cep->mpa.ext_data.bits =0 ;__mpa_ext_set_cc (&cep->mpa.ext_data.bits, qp->attrs.cc); - cep->mpa.ext_data.cookie =cpu_to_be32 (cep->qp->attrs.cookie); + cep->mpa.ext_data.cookie =cpu_to_be32 (cep->qp->attrs.iwarp.cookie); ret =erdma_send_mpareqrep (cep, params->private_data, params->private_data_len);
drivers/infiniband/hw/erdma/erdma_hw.h +1 -0
@@ -31,6 +31,7 @@enum erdma_proto_type {ERDMA_PROTO_IWARP =0 ,ERDMA_PROTO_ROCEV2 =1 , +ERDMA_PROTO_COUNT =2 , };/* PCIe Bar0 Registers. */
drivers/infiniband/hw/erdma/erdma_main.c +1 -2
@@ -486,7 +486,6 @@ static const struct ib_device_ops erdma_device_ops_rocev2 = { .query_pkey = erdma_query_pkey, .create_ah = erdma_create_ah, .destroy_ah = erdma_destroy_ah, - .modify_qp = erdma_modify_qp_rocev2, };static const struct ib_device_ops erdma_device_ops_iwarp = {
@@ -498,7 +497,6 @@ static const struct ib_device_ops erdma_device_ops_iwarp = { .iw_get_qp = erdma_get_ibqp, .iw_reject = erdma_reject, .iw_rem_ref = erdma_qp_put_ref, - .modify_qp = erdma_modify_qp, };static const struct ib_device_ops erdma_device_ops = {
@@ -533,6 +531,7 @@ static const struct ib_device_ops erdma_device_ops = { .query_qp = erdma_query_qp, .req_notify_cq = erdma_req_notify_cq, .reg_user_mr = erdma_reg_user_mr, + .modify_qp = erdma_modify_qp,INIT_RDMA_OBJ_SIZE (ib_cq, erdma_cq, ibcq), INIT_RDMA_OBJ_SIZE(ib_pd, erdma_pd, ibpd),
drivers/infiniband/hw/erdma/erdma_qp.c +80 -56
220 lines not on this page · raw message
drivers/infiniband/hw/erdma/erdma_verbs.c +108 -102
292 lines not on this page · raw message
drivers/infiniband/hw/erdma/erdma_verbs.h +47 -28
119 lines not on this page · raw message
Signature · 1 line
2.43.5
PATCH v2 7/8 RDMA/erdma: Add the query_qp command to the cmdq
In reply to Boshi YuCertian QP attributes, such as sq_draining, can only be obtained by querying the hardware on the erdma RoCEv2 device. To address this, we add the query_qp command to the cmdq and parse the response to retrieve corresponding QP attributes.
--- drivers/infiniband/hw/erdma/erdma_hw.h | 12 +++++ drivers/infiniband/hw/erdma/erdma_verbs.c | 58 +++++++++++++++++++---- 2 files changed, 60 insertions(+), 10 deletions(-)
drivers/infiniband/hw/erdma/erdma_hw.h +12 -0
@@ -154,6 +154,7 @@ enum CMDQ_RDMA_OPCODE {CMDQ_OPCODE_SET_GID =14 ,CMDQ_OPCODE_CREATE_AH =15 ,CMDQ_OPCODE_DESTROY_AH =16 , + CMDQ_OPCODE_QUERY_QP = 17, }; enum CMDQ_COMMON_OPCODE {
@@ -362,6 +363,17 @@ struct erdma_cmdq_mod_qp_req_rocev2 {struct erdma_av_cfg av_cfg; }; +/* query qp response mask */ +#define ERDMA_CMD_QUERY_QP_RESP_SQ_PSN_MASK GENMASK_ULL(23, 0) +#define ERDMA_CMD_QUERY_QP_RESP_RQ_PSN_MASK GENMASK_ULL(47, 24) +#define ERDMA_CMD_QUERY_QP_RESP_QP_STATE_MASK GENMASK_ULL(55, 48) +#define ERDMA_CMD_QUERY_QP_RESP_SQ_DRAINING_MASK GENMASK_ULL(56, 56) + +struct erdma_cmdq_query_qp_req_rocev2 { +u64 hdr; +u32 qpn; +}; +/* create qp cfg0 */ #define ERDMA_CMD_CREATE_QP_SQ_DEPTH_MASK GENMASK(31, 20)#define ERDMA_CMD_CREATE_QP_QPN_MASK GENMASK(19, 0)
drivers/infiniband/hw/erdma/erdma_verbs.c +48 -10
84 lines not on this page · raw message
Signature · 1 line
2.43.5
PATCH v2 8/8 RDMA/erdma: Support UD QPs and UD WRs
In reply to Boshi YuThe iWARP protocol supports only RC QPs previously. Now we add UD QPs and UD WRs support for the RoCEv2 protocol.
--- drivers/infiniband/hw/erdma/erdma_cq.c | 20 +++++++ drivers/infiniband/hw/erdma/erdma_hw.h | 37 +++++++++++- drivers/infiniband/hw/erdma/erdma_qp.c | 71 ++++++++++++++++++----- drivers/infiniband/hw/erdma/erdma_verbs.c | 29 +++++++-- 4 files changed, 136 insertions(+), 21 deletions(-)
drivers/infiniband/hw/erdma/erdma_cq.c +20 -0
@@ -105,6 +105,22 @@ static const struct { { ERDMA_WC_RETRY_EXC_ERR, IB_WC_RETRY_EXC_ERR, ERDMA_WC_VENDOR_NO_ERR }, }; +static void erdma_process_ud_cqe (struct erdma_cqe *cqe,struct ib_wc *wc) +{ +u32 ud_info; + + wc->wc_flags |= (IB_WC_GRH |IB_WC_WITH_NETWORK_HDR_TYPE ); + ud_info =be32_to_cpu (cqe->ud.info); + wc->network_hdr_type =FIELD_GET (ERDMA_CQE_NTYPE_MASK , ud_info); +if (wc->network_hdr_type ==ERDMA_NETWORK_TYPE_IPV4 ) + wc->network_hdr_type =RDMA_NETWORK_IPV4 ; +else + wc->network_hdr_type =RDMA_NETWORK_IPV6 ; + wc->src_qp =FIELD_GET (ERDMA_CQE_SQPN_MASK , ud_info); + wc->sl =FIELD_GET (ERDMA_CQE_SL_MASK , ud_info); + wc->pkey_index =0 ; +} +#define ERDMA_POLLCQ_NO_QP 1static int erdma_poll_one_cqe (struct erdma_cq *cq,struct ib_wc *wc)
@@ -168,6 +184,10 @@ static int erdma_poll_one_cqe(struct erdma_cq *cq, struct ib_wc *wc) wc->wc_flags |=IB_WC_WITH_INVALIDATE ; } +if (erdma_device_rocev2 (dev) && + (qp->ibqp.qp_type ==IB_QPT_UD || qp->ibqp.qp_type ==IB_QPT_GSI )) +erdma_process_ud_cqe (cqe, wc); +if (syndrome >=ERDMA_NUM_WC_STATUS ) syndrome =ERDMA_WC_GENERAL_ERR ;
drivers/infiniband/hw/erdma/erdma_hw.h +35 -2
@@ -374,6 +374,11 @@ struct erdma_cmdq_query_qp_req_rocev2 {u32 qpn; }; +enum erdma_qp_type { +ERDMA_QPT_RC =0 , +ERDMA_QPT_UD =1 , +}; +/* create qp cfg0 */ #define ERDMA_CMD_CREATE_QP_SQ_DEPTH_MASK GENMASK(31, 20)#define ERDMA_CMD_CREATE_QP_QPN_MASK GENMASK(19, 0)
@@ -382,6 +387,9 @@ struct erdma_cmdq_query_qp_req_rocev2 {#define ERDMA_CMD_CREATE_QP_RQ_DEPTH_MASK GENMASK(31, 20)#define ERDMA_CMD_CREATE_QP_PD_MASK GENMASK(19, 0) +/* create qp cfg2 */ +#define ERDMA_CMD_CREATE_QP_TYPE_MASK GENMASK(3, 0) +/* create qp cqn_mtt_cfg */ #define ERDMA_CMD_CREATE_QP_PAGE_SIZE_MASK GENMASK(31, 28)#define ERDMA_CMD_CREATE_QP_DB_CFG_MASK BIT(25)
@@ -415,6 +423,7 @@ struct erdma_cmdq_create_qp_req {u64 rq_mtt_entry[3 ];u32 db_cfg; +u32 cfg2; };struct erdma_cmdq_destroy_qp_req {
@@ -522,6 +531,10 @@ enum {#define ERDMA_CQE_QTYPE_RQ 1#define ERDMA_CQE_QTYPE_CMDQ 2 +#define ERDMA_CQE_NTYPE_MASK BIT(31) +#define ERDMA_CQE_SL_MASK GENMASK(27, 20) +#define ERDMA_CQE_SQPN_MASK GENMASK(19, 0) +struct erdma_cqe {__be32 hdr;__be32 qe_idx;
@@ -531,7 +544,16 @@ struct erdma_cqe {__be32 inv_rkey; };__be32 size; -__be32 rsvd[3 ]; +union { +struct { +__be32 rsvd[3 ]; + } rc; + +struct { +__be32 rsvd[2 ]; +__be32 info; + } ud; + }; };struct erdma_sge {
@@ -583,7 +605,7 @@ struct erdma_write_sqe {struct erdma_sge sgl[]; }; -struct erdma_send_sqe { +struct erdma_send_sqe_rc {__le64 hdr;union {__be32 imm_data;
@@ -594,6 +616,17 @@ struct erdma_send_sqe {struct erdma_sge sgl[]; }; +struct erdma_send_sqe_ud { +__le64 hdr; +__be32 imm_data; +__le32 length; +__le32 qkey; +__le32 dst_qpn; +__le32 ahn; +__le32 rsvd; +struct erdma_sge sgl[]; +}; +struct erdma_readreq_sqe {__le64 hdr;__le32 invalid_stag;
drivers/infiniband/hw/erdma/erdma_qp.c +57 -14
101 lines not on this page · raw message
drivers/infiniband/hw/erdma/erdma_verbs.c +24 -5
@@ -55,6 +55,13 @@ static int create_qp_cmd(struct erdma_ucontext *uctx, struct erdma_qp *qp) ilog2(qp->attrs.rq_size)) |FIELD_PREP (ERDMA_CMD_CREATE_QP_PD_MASK , pd->pdn); +if (qp->ibqp.qp_type ==IB_QPT_RC ) + req.cfg2 =FIELD_PREP (ERDMA_CMD_CREATE_QP_TYPE_MASK , +ERDMA_QPT_RC ); +else + req.cfg2 =FIELD_PREP (ERDMA_CMD_CREATE_QP_TYPE_MASK , +ERDMA_QPT_UD ); +if (rdma_is_kernel_res (&qp->ibqp.res)) {u32 pgsz_range =ilog2 (SZ_1M ) -ERDMA_HW_PAGE_SHIFT ;
@@ -481,7 +488,11 @@ static int erdma_qp_validate_cap(struct erdma_dev *dev,static int erdma_qp_validate_attr (struct erdma_dev *dev,struct ib_qp_init_attr *attrs) { -if (attrs->qp_type !=IB_QPT_RC ) +if (erdma_device_iwarp (dev) && attrs->qp_type !=IB_QPT_RC ) +return -EOPNOTSUPP ; + +if (erdma_device_rocev2 (dev) && attrs->qp_type !=IB_QPT_RC && + attrs->qp_type !=IB_QPT_UD && attrs->qp_type !=IB_QPT_GSI )return -EOPNOTSUPP ;if (attrs->srq)
@@ -959,7 +970,8 @@ int erdma_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs, udata, struct erdma_ucontext, ibucontext);struct erdma_ureq_create_qp ureq;struct erdma_uresp_create_qp uresp; -int ret; +void *old_entry; +int ret =0 ; ret =erdma_qp_validate_cap (dev, attrs);if (ret)
@@ -978,9 +990,16 @@ int erdma_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs,kref_init (&qp->ref);init_completion (&qp->safe_free); - ret =xa_alloc_cyclic (&dev->qp_xa, &qp->ibqp.qp_num, qp, -XA_LIMIT (1 , dev->attrs.max_qp -1 ), - &dev->next_alloc_qpn,GFP_KERNEL ); +if (qp->ibqp.qp_type ==IB_QPT_GSI ) { + old_entry =xa_store (&dev->qp_xa,1 , qp,GFP_KERNEL ); +if (xa_is_err (old_entry)) + ret =xa_err (old_entry); + }else { + ret =xa_alloc_cyclic (&dev->qp_xa, &qp->ibqp.qp_num, qp, +XA_LIMIT (1 , dev->attrs.max_qp -1 ), + &dev->next_alloc_qpn,GFP_KERNEL ); + } +if (ret <0 ) { ret = -ENOMEM ; goto err_out;
Signature · 1 line
2.43.5
Re: [PATCH for-next v2 8/8] RDMA/erdma: Support UD QPs and UD WRs
Op 11-12-2024 om 03:09 schreef Boshi Yu:
Quoted 27 lines
The iWARP protocol supports only RC QPs previously. Now we add UD QPs and UD WRs support for the RoCEv2 protocol.
Signed-off-by Boshi Yu <[email protected]>Reviewed-by Cheng Xu <[email protected]>--- drivers/infiniband/hw/erdma/erdma_cq.c | 20 +++++++ drivers/infiniband/hw/erdma/erdma_hw.h | 37 +++++++++++- drivers/infiniband/hw/erdma/erdma_qp.c | 71 ++++++++++++++++++----- drivers/infiniband/hw/erdma/erdma_verbs.c | 29 +++++++-- 4 files changed, 136 insertions(+), 21 deletions(-)[...]
drivers/infiniband/hw/erdma/erdma_qp.c +6
-0@@ -398,17 +398,57 @@ static int fill_sgl(struct erdma_qp *qp, const struct ib_send_wr *send_wr,return 0 ; } +static void init_send_sqe_rc (struct erdma_qp *qp,struct erdma_send_sqe_rc *sqe, +const struct ib_send_wr *wr,u32 *hw_op) +{ +u32 op =ERDMA_OP_SEND ; + +if (wr->opcode ==IB_WR_SEND_WITH_IMM ) {
drivers/infiniband/hw/erdma/erdma_qp.c static int fill_sgl(struct erdma_qp *qp, const struct ib_send_wr *send_wr, · line 416
Quoted 10 lines
drivers/infiniband/hw/erdma/erdma_qp.c +10
-0+ op =ERDMA_OP_SEND_WITH_IMM ; + sqe->imm_data = wr->ex.imm_data; + }else if (op ==IB_WR_SEND_WITH_INV ) { + op =ERDMA_OP_SEND_WITH_INV ; + sqe->invalid_stag =cpu_to_le32 (wr->ex.invalidate_rkey); + } + + *hw_op = op; +} +
The else if condition is always false. Is there maybe a typo?
Signature · 1 line
Kees
Cheng Xu
maintainerRe: [PATCH for-next v2 8/8] RDMA/erdma: Support UD QPs and UD WRs
Quoted 30 lines · Kees Bakker
Op 11-12-2024 om 03:09 schreef Boshi Yu:
Quoted 29 lines
The iWARP protocol supports only RC QPs previously. Now we add UD QPs and UD WRs support for the RoCEv2 protocol.
Signed-off-by Boshi Yu <[email protected]>Reviewed-by Cheng Xu <[email protected]>---
drivers/infiniband/hw/erdma/erdma_cq.c | 20 +++++++ drivers/infiniband/hw/erdma/erdma_hw.h | 37 +++++++++++- drivers/infiniband/hw/erdma/erdma_qp.c | 71 ++++++++++++++++++----- drivers/infiniband/hw/erdma/erdma_verbs.c | 29 +++++++-- 4 files changed, 136 insertions(+), 21 deletions(-)[...]
drivers/infiniband/hw/erdma/erdma_qp.c +8
-0@@ -398,17 +398,57 @@ static int fill_sgl(struct erdma_qp *qp, const struct ib_send_wr *send_wr, return 0; } +static void init_send_sqe_rc(struct erdma_qp *qp, struct erdma_send_sqe_rc *sqe, + const struct ib_send_wr *wr, u32 *hw_op) +{ + u32 op = ERDMA_OP_SEND; + + if (wr->opcode == IB_WR_SEND_WITH_IMM) { + op = ERDMA_OP_SEND_WITH_IMM; + sqe->imm_data = wr->ex.imm_data; + } else if (op == IB_WR_SEND_WITH_INV) {
Quoted 8 lines
drivers/infiniband/hw/erdma/erdma_qp.c static int fill_sgl(struct erdma_qp *qp, const struct ib_send_wr *send_wr, · line 416
Quoted 7 lines
drivers/infiniband/hw/erdma/erdma_qp.c +7
-0+ op = ERDMA_OP_SEND_WITH_INV; + sqe->invalid_stag = cpu_to_le32(wr->ex.invalidate_rkey); + } + + *hw_op = op; +} +The else if condition is always false. Is there maybe a typo?
Hi, Kees,
Thanks for pointing out this, and it has been fixed by Advait Dhamorikar.
https://lore.kernel.org/lkml/[email protected]/T/
Thanks,
Cheng Xu
Leon Romanovsky
AppliedQuoted 9 lines · Boshi Yu
This patch series introduces support for the RoCEv2 protocol into the erdma driver. As the most prevalent RDMA protocol, RoCEv2 is widely used in the production environment. Given the extensive application of erdma across various scenarios in the Alibaba Cloud, there has arisen a requirement for erdma to support the RoCEv2 protocol. Therefore, we update both the erdma hardware and the erdma driver to accommodate the RoCEv2 protocol.
[...]
Applied, thanks!
[1/8] RDMA/erdma: Probe the erdma RoCEv2 device
https://git.kernel.org/rdma/rdma/c/a883e71345a010 [2/8] RDMA/erdma: Add GID table management interfaces https://git.kernel.org/rdma/rdma/c/6edc15abc256f6 [3/8] RDMA/erdma: Add the erdma_query_pkey() interface https://git.kernel.org/rdma/rdma/c/14bcf7354a0ed2 [4/8] RDMA/erdma: Add address handle implementation https://git.kernel.org/rdma/rdma/c/41dcaf48ff9e31 [5/8] RDMA/erdma: Add erdma_modify_qp_rocev2() interface https://git.kernel.org/rdma/rdma/c/9566cf6a7742f2 [6/8] RDMA/erdma: Refactor the code of the modify_qp interface https://git.kernel.org/rdma/rdma/c/de5b8008aa4da7 [7/8] RDMA/erdma: Add the query_qp command to the cmdq https://git.kernel.org/rdma/rdma/c/1cccbd3eec3d63 [8/8] RDMA/erdma: Support UD QPs and UD WRs https://git.kernel.org/rdma/rdma/c/999a0a2e9b87c4
Best regards,
Signature · 1 line
Leon Romanovsky <[email protected]>