RDMA/hns: Optimize PBL buffer allocation process
Subsystems (2)
v4 cover RDMA/hns: Optimize PBL buffer allocation process
Patch #1 and #2 aim to use MTR interfaces for PBL buffer instead of MTT, and after this, MTT can be removed completely. Patch #3 and #5 refactor buffer size calculation process for WQE and SRQ. #4 can be considered as a preparation for #5, which just moves code of SRQ together to a more suitable place.
This series looks huge, but most of the modification is to replace and remove old interfaces, and patch #4 also contribute a lot. Actually, the original logic is not changed so much.
Changes since v3:
- Fix a sparce warning about a function that should be static reported by kbuild test robot.
Changes since v2:
- Just do a rebase to current for-next branch.
Changes since v1:
- Remove meaningless judgment of count in some inline functions in #3.
- Add more information into commit messages of #3 and #5.
Xi Wang (4):
RDMA/hns: Optimize PBL buffer allocation process RDMA/hns: Remove unused MTT functions RDMA/hns: Optimize WQE buffer size calculating process RDMA/hns: Optimize SRQ buffer size calculating process
Yixian Liu (1):
RDMA/hns: Move SRQ code to the reasonable place
drivers/infiniband/hw/hns/hns_roce_alloc.c | 43 - drivers/infiniband/hw/hns/hns_roce_device.h | 110 +-- drivers/infiniband/hw/hns/hns_roce_hem.c | 105 --- drivers/infiniband/hw/hns/hns_roce_hem.h | 6 - drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 45 +- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 943 ++++++++++---------- drivers/infiniband/hw/hns/hns_roce_main.c | 70 +- drivers/infiniband/hw/hns/hns_roce_mr.c | 1247 +++------------------------ drivers/infiniband/hw/hns/hns_roce_qp.c | 313 +++---- drivers/infiniband/hw/hns/hns_roce_srq.c | 16 +- 10 files changed, 767 insertions(+), 2131 deletions(-)
Signature · 1 line
2.8.1
PATCH v4 1/5 RDMA/hns: Optimize PBL buffer allocation process
From: Xi Wang <[email protected]>
PBL table has its own implementation for multi-hop addressing currently, but for the hardware, all table's addressing use the same logic, there is no need to implement repeatedly. So optimize the PBL buffer allocation process by using the mtr's interfaces.
--- drivers/infiniband/hw/hns/hns_roce_device.h | 19 +- drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 45 +- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 76 +-- drivers/infiniband/hw/hns/hns_roce_mr.c | 726 ++++++---------------------- 4 files changed, 197 insertions(+), 669 deletions(-)
drivers/infiniband/hw/hns/hns_roce_device.h +3 -16
@@ -403,30 +403,17 @@ struct hns_roce_mw {struct hns_roce_mr {struct ib_mr ibmr; -struct ib_umem *umem;u64 iova;/* MR's virtual orignal addr */ u64 size;/* Address range of MR */ u32 key;/* Key of MR */ u32 pd;/* PD num of MR */ u32 access;/* Access permission of MR */ -u32 npages;int enabled;/* MR's active status */ int type;/* MR's register type */ -u64 *pbl_buf;/* MR's PBL space */ -dma_addr_t pbl_dma_addr;/* MR's PBL space PA */ -u32 pbl_size;/* PA number in the PBL */ -u64 pbl_ba;/* page table address */ -u32 l0_chunk_last_num;/* L0 last number */ -u32 l1_chunk_last_num;/* L1 last number */ -u64 **pbl_bt_l2;/* PBL BT L2 */ -u64 **pbl_bt_l1;/* PBL BT L1 */ -u64 *pbl_bt_l0;/* PBL BT L0 */ -dma_addr_t *pbl_l2_dma_addr;/* PBL BT L2 dma addr */ -dma_addr_t *pbl_l1_dma_addr;/* PBL BT L1 dma addr */ -dma_addr_t pbl_l0_dma_addr;/* PBL BT L0 dma addr */ -u32 pbl_ba_pg_sz;/* BT chunk page size */ -u32 pbl_buf_pg_sz;/* buf chunk page size */ u32 pbl_hop_num;/* multi-hop number */ +struct hns_roce_mtr pbl_mtr; +u32 npages; +dma_addr_t *page_list; };struct hns_roce_mr_table {
drivers/infiniband/hw/hns/hns_roce_hw_v1.c +14 -31
82 lines not on this page · raw message
drivers/infiniband/hw/hns/hns_roce_hw_v2.c +41 -35
149 lines not on this page · raw message
drivers/infiniband/hw/hns/hns_roce_mr.c +139 -587
921 lines not on this page · raw message
Signature · 1 line
2.8.1
PATCH v4 2/5 RDMA/hns: Remove unused MTT functions
In reply to Weihang LiFrom: Xi Wang <[email protected]>
The MTT (Memory Translate Table) interface is no longer used to configure the buffer address to BT (Base Address Table) that requires driver mapping. Because the MTT is not compatible with multi-hop addressing of the hip08, it is replaced by MTR (Memory Translate Region) interface, and all the MTT functions should be removed.
--- drivers/infiniband/hw/hns/hns_roce_alloc.c | 43 --- drivers/infiniband/hw/hns/hns_roce_device.h | 64 ---- drivers/infiniband/hw/hns/hns_roce_hem.c | 105 ------ drivers/infiniband/hw/hns/hns_roce_hem.h | 6 - drivers/infiniband/hw/hns/hns_roce_main.c | 70 +--- drivers/infiniband/hw/hns/hns_roce_mr.c | 521 ---------------------------- 6 files changed, 2 insertions(+), 807 deletions(-)
drivers/infiniband/hw/hns/hns_roce_alloc.c +0 -43
@@ -283,49 +283,6 @@ int hns_roce_get_umem_bufs(struct hns_roce_dev *hr_dev, dma_addr_t *bufs,return total; } -void hns_roce_init_buf_region (struct hns_roce_buf_region *region,int hopnum, -int offset,int buf_cnt) -{ -if (hopnum ==HNS_ROCE_HOP_NUM_0 ) - region->hopnum =0 ; -else - region->hopnum = hopnum; - - region->offset = offset; - region->count = buf_cnt; -} - -void hns_roce_free_buf_list (dma_addr_t **bufs,int region_cnt) -{ -int i; - -for (i =0 ; i < region_cnt; i++) { -kfree (bufs[i]); - bufs[i] =NULL ; - } -} - -int hns_roce_alloc_buf_list (struct hns_roce_buf_region *regions, -dma_addr_t **bufs,int region_cnt) -{ -struct hns_roce_buf_region *r; -int i; - -for (i =0 ; i < region_cnt; i++) { - r = ®ions[i]; - bufs[i] =kcalloc (r->count,sizeof (dma_addr_t),GFP_KERNEL ); -if (!bufs[i]) - goto err_alloc; - } - -return 0 ; - -err_alloc: -hns_roce_free_buf_list (bufs, i); - -return -ENOMEM ; -} -void hns_roce_cleanup_bitmap (struct hns_roce_dev *hr_dev) {if (hr_dev->caps.flags &HNS_ROCE_CAP_FLAG_SRQ )
drivers/infiniband/hw/hns/hns_roce_device.h +0 -64
107 lines not on this page · raw message
drivers/infiniband/hw/hns/hns_roce_hem.c +0 -105
135 lines not on this page · raw message
drivers/infiniband/hw/hns/hns_roce_hem.h +0 -6
@@ -115,12 +115,6 @@ void hns_roce_table_put(struct hns_roce_dev *hr_dev,void *hns_roce_table_find (struct hns_roce_dev *hr_dev,struct hns_roce_hem_table *table,unsigned long obj,dma_addr_t *dma_handle); -int hns_roce_table_get_range (struct hns_roce_dev *hr_dev, -struct hns_roce_hem_table *table, -unsigned long start,unsigned long end); -void hns_roce_table_put_range (struct hns_roce_dev *hr_dev, -struct hns_roce_hem_table *table, -unsigned long start,unsigned long end); int hns_roce_init_hem_table(struct hns_roce_dev *hr_dev, struct hns_roce_hem_table *table, u32 type, unsigned long obj_size, unsigned long nobj,
drivers/infiniband/hw/hns/hns_roce_main.c +2 -68
105 lines not on this page · raw message
drivers/infiniband/hw/hns/hns_roce_mr.c +0 -521
563 lines not on this page · raw message
Signature · 1 line
2.8.1
PATCH v4 3/5 RDMA/hns: Optimize WQE buffer size calculating process
In reply to Weihang LiFrom: Xi Wang <[email protected]>
Optimize the QP's WQE buffer parameters calculating process to make the codes more readable mainly by merging calculation of extended sge space of kernel and userspace. In addition, add some inline functions to simply codes about multi-hop addressing.
--- drivers/infiniband/hw/hns/hns_roce_device.h | 25 +++ drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 109 ++++------ drivers/infiniband/hw/hns/hns_roce_qp.c | 313 +++++++++++----------------- 3 files changed, 182 insertions(+), 265 deletions(-)
drivers/infiniband/hw/hns/hns_roce_device.h +25 -0
@@ -1079,6 +1079,8 @@ static inline dma_addr_t hns_roce_buf_page(struct hns_roce_buf *buf, int idx)return buf->page_list[idx].map; } +#define hr_hw_page_align (x) ALIGN(x, 1 << PAGE_ADDR_SHIFT) +static inline u64 to_hr_hw_page_addr (u64 addr) {return addr >>PAGE_ADDR_SHIFT ;
@@ -1089,6 +1091,29 @@ static inline u32 to_hr_hw_page_shift(u32 page_shift)return page_shift -PAGE_ADDR_SHIFT ; } +static inline u32 to_hr_hem_hopnum (u32 hopnum,u32 count) +{ +if (count >0 ) +return hopnum ==HNS_ROCE_HOP_NUM_0 ?0 : hopnum; + +return 0 ; +} + +static inline u32 to_hr_hem_entries_size (u32 count,u32 buf_shift) +{ +return hr_hw_page_align (count << buf_shift); +} + +static inline u32 to_hr_hem_entries_count (u32 count,u32 buf_shift) +{ +return hr_hw_page_align (count << buf_shift) >> buf_shift; +} + +static inline u32 to_hr_hem_entries_shift (u32 count,u32 buf_shift) +{ +return ilog2 (to_hr_hem_entries_count (count, buf_shift)); +} +int hns_roce_init_uar_table (struct hns_roce_dev *dev);int hns_roce_uar_alloc (struct hns_roce_dev *dev,struct hns_roce_uar *uar);void hns_roce_uar_free (struct hns_roce_dev *dev,struct hns_roce_uar *uar);
drivers/infiniband/hw/hns/hns_roce_hw_v2.c +36 -73
181 lines not on this page · raw message
drivers/infiniband/hw/hns/hns_roce_qp.c +121 -192
471 lines not on this page · raw message
Signature · 1 line
2.8.1
PATCH v4 4/5 RDMA/hns: Move SRQ code to the reasonable place
In reply to Weihang LiFrom: Yixian Liu <[email protected]>
Just move the SRQ related code to more reasonable place, and unify format of some prints.
--- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 795 ++++++++++++++--------------- 1 file changed, 396 insertions(+), 399 deletions(-)
drivers/infiniband/hw/hns/hns_roce_hw_v2.c +396 -399
@@ -694,6 +694,129 @@ static int hns_roce_v2_post_recv(struct ib_qp *ibqp,return ret; } +static void *get_srq_wqe (struct hns_roce_srq *srq,int n) +{ +return hns_roce_buf_offset (srq->buf_mtr.kmem, n << srq->wqe_shift); +} + +static void hns_roce_free_srq_wqe (struct hns_roce_srq *srq,int wqe_index) +{ +/* always called with interrupts disabled. */ +spin_lock (&srq->lock); + +bitmap_clear (srq->idx_que.bitmap, wqe_index,1 ); + srq->tail++; + +spin_unlock (&srq->lock); +} + +static int find_empty_entry (struct hns_roce_idx_que *idx_que, +unsigned long size) +{ +int wqe_idx; + +if (unlikely (bitmap_full (idx_que->bitmap, size))) +return -ENOSPC ; + + wqe_idx =find_first_zero_bit (idx_que->bitmap, size); + +bitmap_set (idx_que->bitmap, wqe_idx,1 ); + +return wqe_idx; +} + +static void fill_idx_queue (struct hns_roce_idx_que *idx_que, +int cur_idx,int wqe_idx) +{ +unsigned int *addr; + + addr = (unsigned int *)hns_roce_buf_offset (idx_que->mtr.kmem, + cur_idx * idx_que->entry_sz); + *addr = wqe_idx; +} + +static int hns_roce_v2_post_srq_recv (struct ib_srq *ibsrq, +const struct ib_recv_wr *wr, +const struct ib_recv_wr **bad_wr) +{ +struct hns_roce_dev *hr_dev =to_hr_dev (ibsrq->device); +struct hns_roce_srq *srq =to_hr_srq (ibsrq); +struct hns_roce_v2_wqe_data_seg *dseg; +struct hns_roce_v2_db srq_db; +unsigned long flags; +int ret =0 ; +int wqe_idx; +void *wqe; +int nreq; +int ind; +int i; + +spin_lock_irqsave (&srq->lock, flags); + + ind = srq->head & (srq->wqe_cnt -1 ); + +for (nreq =0 ; wr; ++nreq, wr = wr->next) { +if (unlikely (wr->num_sge >= srq->max_gs)) { + ret = -EINVAL ; + *bad_wr = wr; +break ; + } + +if (unlikely (srq->head == srq->tail)) { + ret = -ENOMEM ; + *bad_wr = wr; +break ; + } + + wqe_idx =find_empty_entry (&srq->idx_que, srq->wqe_cnt); +if (wqe_idx <0 ) { + ret = -ENOMEM ; + *bad_wr = wr; +break ; + } + +fill_idx_queue (&srq->idx_que, ind, wqe_idx); + wqe =get_srq_wqe (srq, wqe_idx); + dseg = (struct hns_roce_v2_wqe_data_seg *)wqe; + +for (i =0 ; i < wr->num_sge; ++i) { + dseg[i].len =cpu_to_le32 (wr->sg_list[i].length); + dseg[i].lkey =cpu_to_le32 (wr->sg_list[i].lkey); + dseg[i].addr =cpu_to_le64 (wr->sg_list[i].addr); + } + +if (i < srq->max_gs) { + dseg[i].len =0 ; + dseg[i].lkey =cpu_to_le32 (0x100 ); + dseg[i].addr =0 ; + } + + srq->wrid[wqe_idx] = wr->wr_id; + ind = (ind +1 ) & (srq->wqe_cnt -1 ); + } + +if (likely (nreq)) { + srq->head += nreq; + +/* +* Make sure that descriptors are written before +* doorbell record. +*/ +wmb (); + + srq_db.byte_4 = +cpu_to_le32 (HNS_ROCE_V2_SRQ_DB <<V2_DB_BYTE_4_CMD_S | + (srq->srqn &V2_DB_BYTE_4_TAG_M )); + srq_db.parameter =cpu_to_le32 (srq->head); + +hns_roce_write64 (hr_dev, (__le32 *)&srq_db, srq->db_reg_l); + } + +spin_unlock_irqrestore (&srq->lock, flags); + +return ret; +} +static int hns_roce_v2_cmd_hw_reseted (struct hns_roce_dev *hr_dev,unsigned long instance_stage,unsigned long reset_stage)
@@ -2667,22 +2790,6 @@ static struct hns_roce_v2_cqe *next_cqe_sw_v2(struct hns_roce_cq *hr_cq)return get_sw_cqe_v2 (hr_cq, hr_cq->cons_index); } -static void *get_srq_wqe (struct hns_roce_srq *srq,int n) -{ -return hns_roce_buf_offset (srq->buf_mtr.kmem, n << srq->wqe_shift); -} - -static void hns_roce_free_srq_wqe (struct hns_roce_srq *srq,int wqe_index) -{ -/* always called with interrupts disabled. */ -spin_lock (&srq->lock); - -bitmap_clear (srq->idx_que.bitmap, wqe_index,1 ); - srq->tail++; - -spin_unlock (&srq->lock); -} -static void hns_roce_v2_cq_set_ci (struct hns_roce_cq *hr_cq,u32 cons_index) { *hr_cq->set_ci_db = cons_index &V2_CQ_DB_PARAMETER_CONS_IDX_M ;
@@ -4777,108 +4884,288 @@ static int hns_roce_v2_qp_flow_control_init(struct hns_roce_dev *hr_dev,return ret; } -static int hns_roce_v2_modify_cq (struct ib_cq *cq,u16 cq_count,u16 cq_period) +static void hns_roce_v2_write_srqc (struct hns_roce_dev *hr_dev, +struct hns_roce_srq *srq,u32 pdn,u16 xrcd, +u32 cqn,void *mb_buf,u64 *mtts_wqe, +u64 *mtts_idx,dma_addr_t dma_handle_wqe, +dma_addr_t dma_handle_idx) { -struct hns_roce_dev *hr_dev =to_hr_dev (cq->device); -struct hns_roce_v2_cq_context *cq_context; -struct hns_roce_cq *hr_cq =to_hr_cq (cq); -struct hns_roce_v2_cq_context *cqc_mask; -struct hns_roce_cmd_mailbox *mailbox; -int ret; - - mailbox =hns_roce_alloc_cmd_mailbox (hr_dev); -if (IS_ERR (mailbox)) -return PTR_ERR (mailbox); +struct hns_roce_srq_context *srq_context; - cq_context = mailbox->buf; - cqc_mask = (struct hns_roce_v2_cq_context *)mailbox->buf +1 ; + srq_context = mb_buf; +memset (srq_context,0 ,sizeof (*srq_context)); -memset (cqc_mask,0xff ,sizeof (*cqc_mask)); +roce_set_field (srq_context->byte_4_srqn_srqst,SRQC_BYTE_4_SRQ_ST_M , +SRQC_BYTE_4_SRQ_ST_S ,1 ); -roce_set_field (cq_context->byte_56_cqe_period_maxcnt, -V2_CQC_BYTE_56_CQ_MAX_CNT_M ,V2_CQC_BYTE_56_CQ_MAX_CNT_S , - cq_count); -roce_set_field (cqc_mask->byte_56_cqe_period_maxcnt, -V2_CQC_BYTE_56_CQ_MAX_CNT_M ,V2_CQC_BYTE_56_CQ_MAX_CNT_S , -0 ); -roce_set_field (cq_context->byte_56_cqe_period_maxcnt, -V2_CQC_BYTE_56_CQ_PERIOD_M ,V2_CQC_BYTE_56_CQ_PERIOD_S , - cq_period); -roce_set_field (cqc_mask->byte_56_cqe_period_maxcnt, -V2_CQC_BYTE_56_CQ_PERIOD_M ,V2_CQC_BYTE_56_CQ_PERIOD_S , -0 ); +roce_set_field (srq_context->byte_4_srqn_srqst, +SRQC_BYTE_4_SRQ_WQE_HOP_NUM_M , +SRQC_BYTE_4_SRQ_WQE_HOP_NUM_S , + (hr_dev->caps.srqwqe_hop_num ==HNS_ROCE_HOP_NUM_0 ?0 : + hr_dev->caps.srqwqe_hop_num)); +roce_set_field (srq_context->byte_4_srqn_srqst, +SRQC_BYTE_4_SRQ_SHIFT_M ,SRQC_BYTE_4_SRQ_SHIFT_S , +ilog2 (srq->wqe_cnt)); - ret =hns_roce_cmd_mbox (hr_dev, mailbox->dma,0 , hr_cq->cqn,1 , -HNS_ROCE_CMD_MODIFY_CQC , -HNS_ROCE_CMD_TIMEOUT_MSECS ); -hns_roce_free_cmd_mailbox (hr_dev, mailbox); -if (ret) -ibdev_err (&hr_dev->ib_dev, -"failed to process cmd when modifying CQ, ret = %d\n" , - ret); +roce_set_field (srq_context->byte_4_srqn_srqst,SRQC_BYTE_4_SRQN_M , +SRQC_BYTE_4_SRQN_S , srq->srqn); -return ret; -} +roce_set_field (srq_context->byte_8_limit_wl,SRQC_BYTE_8_SRQ_LIMIT_WL_M , +SRQC_BYTE_8_SRQ_LIMIT_WL_S ,0 ); -static void hns_roce_irq_work_handle (struct work_struct *work) -{ -struct hns_roce_work *irq_work = - container_of(work, struct hns_roce_work, work); -struct ib_device *ibdev = &irq_work->hr_dev->ib_dev; -u32 qpn = irq_work->qpn; -u32 cqn = irq_work->cqn; +roce_set_field (srq_context->byte_12_xrcd,SRQC_BYTE_12_SRQ_XRCD_M , +SRQC_BYTE_12_SRQ_XRCD_S , xrcd); -switch (irq_work->event_type) { -case HNS_ROCE_EVENT_TYPE_PATH_MIG : -ibdev_info (ibdev,"Path migrated succeeded.\n" ); -break ; -case HNS_ROCE_EVENT_TYPE_PATH_MIG_FAILED : -ibdev_warn (ibdev,"Path migration failed.\n" ); -break ; -case HNS_ROCE_EVENT_TYPE_COMM_EST : -break ; -case HNS_ROCE_EVENT_TYPE_SQ_DRAINED : -ibdev_warn (ibdev,"Send queue drained.\n" ); -break ; -case HNS_ROCE_EVENT_TYPE_WQ_CATAS_ERROR : -ibdev_err (ibdev,"Local work queue 0x%x catast error, sub_event type is: %d\n" , - qpn, irq_work->sub_type); -break ; -case HNS_ROCE_EVENT_TYPE_INV_REQ_LOCAL_WQ_ERROR : -ibdev_err (ibdev,"Invalid request local work queue 0x%x error.\n" , - qpn); -break ; -case HNS_ROCE_EVENT_TYPE_LOCAL_WQ_ACCESS_ERROR : -ibdev_err (ibdev,"Local access violation work queue 0x%x error, sub_event type is: %d\n" , - qpn, irq_work->sub_type); -break ; -case HNS_ROCE_EVENT_TYPE_SRQ_LIMIT_REACH : -ibdev_warn (ibdev,"SRQ limit reach.\n" ); -break ; -case HNS_ROCE_EVENT_TYPE_SRQ_LAST_WQE_REACH : -ibdev_warn (ibdev,"SRQ last wqe reach.\n" ); -break ; -case HNS_ROCE_EVENT_TYPE_SRQ_CATAS_ERROR : -ibdev_err (ibdev,"SRQ catas error.\n" ); -break ; -case HNS_ROCE_EVENT_TYPE_CQ_ACCESS_ERROR : -ibdev_err (ibdev,"CQ 0x%x access err.\n" , cqn); -break ; -case HNS_ROCE_EVENT_TYPE_CQ_OVERFLOW : -ibdev_warn (ibdev,"CQ 0x%x overflow\n" , cqn); -break ; -case HNS_ROCE_EVENT_TYPE_DB_OVERFLOW : -ibdev_warn (ibdev,"DB overflow.\n" ); -break ; -case HNS_ROCE_EVENT_TYPE_FLR : -ibdev_warn (ibdev,"Function level reset.\n" ); -break ; -default : -break ; - } + srq_context->wqe_bt_ba =cpu_to_le32 ((u32)(dma_handle_wqe >>3 )); -kfree (irq_work); -} +roce_set_field (srq_context->byte_24_wqe_bt_ba, +SRQC_BYTE_24_SRQ_WQE_BT_BA_M , +SRQC_BYTE_24_SRQ_WQE_BT_BA_S , + dma_handle_wqe >>35 ); + +roce_set_field (srq_context->byte_28_rqws_pd,SRQC_BYTE_28_PD_M , +SRQC_BYTE_28_PD_S , pdn); +roce_set_field (srq_context->byte_28_rqws_pd,SRQC_BYTE_28_RQWS_M , +SRQC_BYTE_28_RQWS_S , srq->max_gs <=0 ?0 : +fls (srq->max_gs -1 )); + + srq_context->idx_bt_ba =cpu_to_le32 (dma_handle_idx >>3 ); +roce_set_field (srq_context->rsv_idx_bt_ba, +SRQC_BYTE_36_SRQ_IDX_BT_BA_M , +SRQC_BYTE_36_SRQ_IDX_BT_BA_S , + dma_handle_idx >>35 ); + + srq_context->idx_cur_blk_addr = +cpu_to_le32 (to_hr_hw_page_addr (mtts_idx[0 ])); +roce_set_field (srq_context->byte_44_idxbufpgsz_addr, +SRQC_BYTE_44_SRQ_IDX_CUR_BLK_ADDR_M , +SRQC_BYTE_44_SRQ_IDX_CUR_BLK_ADDR_S , +upper_32_bits (to_hr_hw_page_addr (mtts_idx[0 ]))); +roce_set_field (srq_context->byte_44_idxbufpgsz_addr, +SRQC_BYTE_44_SRQ_IDX_HOP_NUM_M , +SRQC_BYTE_44_SRQ_IDX_HOP_NUM_S , + hr_dev->caps.idx_hop_num ==HNS_ROCE_HOP_NUM_0 ?0 : + hr_dev->caps.idx_hop_num); + +roce_set_field (srq_context->byte_44_idxbufpgsz_addr, +SRQC_BYTE_44_SRQ_IDX_BA_PG_SZ_M , +SRQC_BYTE_44_SRQ_IDX_BA_PG_SZ_S , +to_hr_hw_page_shift (srq->idx_que.mtr.hem_cfg.ba_pg_shift)); +roce_set_field (srq_context->byte_44_idxbufpgsz_addr, +SRQC_BYTE_44_SRQ_IDX_BUF_PG_SZ_M , +SRQC_BYTE_44_SRQ_IDX_BUF_PG_SZ_S , +to_hr_hw_page_shift (srq->idx_que.mtr.hem_cfg.buf_pg_shift)); + + srq_context->idx_nxt_blk_addr = +cpu_to_le32 (to_hr_hw_page_addr (mtts_idx[1 ])); +roce_set_field (srq_context->rsv_idxnxtblkaddr, +SRQC_BYTE_52_SRQ_IDX_NXT_BLK_ADDR_M , +SRQC_BYTE_52_SRQ_IDX_NXT_BLK_ADDR_S , +upper_32_bits (to_hr_hw_page_addr (mtts_idx[1 ]))); +roce_set_field (srq_context->byte_56_xrc_cqn, +SRQC_BYTE_56_SRQ_XRC_CQN_M ,SRQC_BYTE_56_SRQ_XRC_CQN_S , + cqn); +roce_set_field (srq_context->byte_56_xrc_cqn, +SRQC_BYTE_56_SRQ_WQE_BA_PG_SZ_M , +SRQC_BYTE_56_SRQ_WQE_BA_PG_SZ_S , +to_hr_hw_page_shift (srq->buf_mtr.hem_cfg.ba_pg_shift)); +roce_set_field (srq_context->byte_56_xrc_cqn, +SRQC_BYTE_56_SRQ_WQE_BUF_PG_SZ_M , +SRQC_BYTE_56_SRQ_WQE_BUF_PG_SZ_S , +to_hr_hw_page_shift (srq->buf_mtr.hem_cfg.buf_pg_shift)); + +roce_set_bit (srq_context->db_record_addr_record_en, +SRQC_BYTE_60_SRQ_RECORD_EN_S ,0 ); +} + +static int hns_roce_v2_modify_srq (struct ib_srq *ibsrq, +struct ib_srq_attr *srq_attr, +enum ib_srq_attr_mask srq_attr_mask, +struct ib_udata *udata) +{ +struct hns_roce_dev *hr_dev =to_hr_dev (ibsrq->device); +struct hns_roce_srq *srq =to_hr_srq (ibsrq); +struct hns_roce_srq_context *srq_context; +struct hns_roce_srq_context *srqc_mask; +struct hns_roce_cmd_mailbox *mailbox; +int ret; + +if (srq_attr_mask &IB_SRQ_LIMIT ) { +if (srq_attr->srq_limit >= srq->wqe_cnt) +return -EINVAL ; + + mailbox =hns_roce_alloc_cmd_mailbox (hr_dev); +if (IS_ERR (mailbox)) +return PTR_ERR (mailbox); + + srq_context = mailbox->buf; + srqc_mask = (struct hns_roce_srq_context *)mailbox->buf +1 ; + +memset (srqc_mask,0xff ,sizeof (*srqc_mask)); + +roce_set_field (srq_context->byte_8_limit_wl, +SRQC_BYTE_8_SRQ_LIMIT_WL_M , +SRQC_BYTE_8_SRQ_LIMIT_WL_S , srq_attr->srq_limit); +roce_set_field (srqc_mask->byte_8_limit_wl, +SRQC_BYTE_8_SRQ_LIMIT_WL_M , +SRQC_BYTE_8_SRQ_LIMIT_WL_S ,0 ); + + ret =hns_roce_cmd_mbox (hr_dev, mailbox->dma,0 , srq->srqn,0 , +HNS_ROCE_CMD_MODIFY_SRQC , +HNS_ROCE_CMD_TIMEOUT_MSECS ); +hns_roce_free_cmd_mailbox (hr_dev, mailbox); +if (ret) { +ibdev_err (&hr_dev->ib_dev, +"failed to handle cmd of modifying SRQ, ret = %d.\n" , + ret); +return ret; + } + } + +return 0 ; +} + +static int hns_roce_v2_query_srq (struct ib_srq *ibsrq,struct ib_srq_attr *attr) +{ +struct hns_roce_dev *hr_dev =to_hr_dev (ibsrq->device); +struct hns_roce_srq *srq =to_hr_srq (ibsrq); +struct hns_roce_srq_context *srq_context; +struct hns_roce_cmd_mailbox *mailbox; +int limit_wl; +int ret; + + mailbox =hns_roce_alloc_cmd_mailbox (hr_dev); +if (IS_ERR (mailbox)) +return PTR_ERR (mailbox); + + srq_context = mailbox->buf; + ret =hns_roce_cmd_mbox (hr_dev,0 , mailbox->dma, srq->srqn,0 , +HNS_ROCE_CMD_QUERY_SRQC , +HNS_ROCE_CMD_TIMEOUT_MSECS ); +if (ret) { +ibdev_err (&hr_dev->ib_dev, +"failed to process cmd of querying SRQ, ret = %d.\n" , + ret); + goto out; + } + + limit_wl =roce_get_field (srq_context->byte_8_limit_wl, +SRQC_BYTE_8_SRQ_LIMIT_WL_M , +SRQC_BYTE_8_SRQ_LIMIT_WL_S ); + + attr->srq_limit = limit_wl; + attr->max_wr = srq->wqe_cnt; + attr->max_sge = srq->max_gs; + +memcpy (srq_context, mailbox->buf,sizeof (*srq_context)); + +out: +hns_roce_free_cmd_mailbox (hr_dev, mailbox); +return ret; +} + +static int hns_roce_v2_modify_cq (struct ib_cq *cq,u16 cq_count,u16 cq_period) +{ +struct hns_roce_dev *hr_dev =to_hr_dev (cq->device); +struct hns_roce_v2_cq_context *cq_context; +struct hns_roce_cq *hr_cq =to_hr_cq (cq); +struct hns_roce_v2_cq_context *cqc_mask; +struct hns_roce_cmd_mailbox *mailbox; +int ret; + + mailbox =hns_roce_alloc_cmd_mailbox (hr_dev); +if (IS_ERR (mailbox)) +return PTR_ERR (mailbox); + + cq_context = mailbox->buf; + cqc_mask = (struct hns_roce_v2_cq_context *)mailbox->buf +1 ; + +memset (cqc_mask,0xff ,sizeof (*cqc_mask)); + +roce_set_field (cq_context->byte_56_cqe_period_maxcnt, +V2_CQC_BYTE_56_CQ_MAX_CNT_M ,V2_CQC_BYTE_56_CQ_MAX_CNT_S , + cq_count); +roce_set_field (cqc_mask->byte_56_cqe_period_maxcnt, +V2_CQC_BYTE_56_CQ_MAX_CNT_M ,V2_CQC_BYTE_56_CQ_MAX_CNT_S , +0 ); +roce_set_field (cq_context->byte_56_cqe_period_maxcnt, +V2_CQC_BYTE_56_CQ_PERIOD_M ,V2_CQC_BYTE_56_CQ_PERIOD_S , + cq_period); +roce_set_field (cqc_mask->byte_56_cqe_period_maxcnt, +V2_CQC_BYTE_56_CQ_PERIOD_M ,V2_CQC_BYTE_56_CQ_PERIOD_S , +0 ); + + ret =hns_roce_cmd_mbox (hr_dev, mailbox->dma,0 , hr_cq->cqn,1 , +HNS_ROCE_CMD_MODIFY_CQC , +HNS_ROCE_CMD_TIMEOUT_MSECS ); +hns_roce_free_cmd_mailbox (hr_dev, mailbox); +if (ret) +ibdev_err (&hr_dev->ib_dev, +"failed to process cmd when modifying CQ, ret = %d\n" , + ret); + +return ret; +} + +static void hns_roce_irq_work_handle (struct work_struct *work) +{ +struct hns_roce_work *irq_work = + container_of(work, struct hns_roce_work, work); +struct ib_device *ibdev = &irq_work->hr_dev->ib_dev; +u32 qpn = irq_work->qpn; +u32 cqn = irq_work->cqn; + +switch (irq_work->event_type) { +case HNS_ROCE_EVENT_TYPE_PATH_MIG : +ibdev_info (ibdev,"Path migrated succeeded.\n" ); +break ; +case HNS_ROCE_EVENT_TYPE_PATH_MIG_FAILED : +ibdev_warn (ibdev,"Path migration failed.\n" ); +break ; +case HNS_ROCE_EVENT_TYPE_COMM_EST : +break ; +case HNS_ROCE_EVENT_TYPE_SQ_DRAINED : +ibdev_warn (ibdev,"Send queue drained.\n" ); +break ; +case HNS_ROCE_EVENT_TYPE_WQ_CATAS_ERROR : +ibdev_err (ibdev,"Local work queue 0x%x catast error, sub_event type is: %d\n" , + qpn, irq_work->sub_type); +break ; +case HNS_ROCE_EVENT_TYPE_INV_REQ_LOCAL_WQ_ERROR : +ibdev_err (ibdev,"Invalid request local work queue 0x%x error.\n" , + qpn); +break ; +case HNS_ROCE_EVENT_TYPE_LOCAL_WQ_ACCESS_ERROR : +ibdev_err (ibdev,"Local access violation work queue 0x%x error, sub_event type is: %d\n" , + qpn, irq_work->sub_type); +break ; +case HNS_ROCE_EVENT_TYPE_SRQ_LIMIT_REACH : +ibdev_warn (ibdev,"SRQ limit reach.\n" ); +break ; +case HNS_ROCE_EVENT_TYPE_SRQ_LAST_WQE_REACH : +ibdev_warn (ibdev,"SRQ last wqe reach.\n" ); +break ; +case HNS_ROCE_EVENT_TYPE_SRQ_CATAS_ERROR : +ibdev_err (ibdev,"SRQ catas error.\n" ); +break ; +case HNS_ROCE_EVENT_TYPE_CQ_ACCESS_ERROR : +ibdev_err (ibdev,"CQ 0x%x access err.\n" , cqn); +break ; +case HNS_ROCE_EVENT_TYPE_CQ_OVERFLOW : +ibdev_warn (ibdev,"CQ 0x%x overflow\n" , cqn); +break ; +case HNS_ROCE_EVENT_TYPE_DB_OVERFLOW : +ibdev_warn (ibdev,"DB overflow.\n" ); +break ; +case HNS_ROCE_EVENT_TYPE_FLR : +ibdev_warn (ibdev,"Function level reset.\n" ); +break ; +default : +break ; + } + +kfree (irq_work); +} static void hns_roce_v2_init_irq_work(struct hns_roce_dev *hr_dev, struct hns_roce_eq *eq,
@@ -5588,296 +5875,6 @@ static void hns_roce_v2_cleanup_eq_table(struct hns_roce_dev *hr_dev)destroy_workqueue (hr_dev->irq_workq); } -static void hns_roce_v2_write_srqc (struct hns_roce_dev *hr_dev, -struct hns_roce_srq *srq,u32 pdn,u16 xrcd, -u32 cqn,void *mb_buf,u64 *mtts_wqe, -u64 *mtts_idx,dma_addr_t dma_handle_wqe, -dma_addr_t dma_handle_idx) -{ -struct hns_roce_srq_context *srq_context; - - srq_context = mb_buf; -memset (srq_context,0 ,sizeof (*srq_context)); - -roce_set_field (srq_context->byte_4_srqn_srqst,SRQC_BYTE_4_SRQ_ST_M , -SRQC_BYTE_4_SRQ_ST_S ,1 ); - -roce_set_field (srq_context->byte_4_srqn_srqst, -SRQC_BYTE_4_SRQ_WQE_HOP_NUM_M , -SRQC_BYTE_4_SRQ_WQE_HOP_NUM_S , - (hr_dev->caps.srqwqe_hop_num ==HNS_ROCE_HOP_NUM_0 ?0 : - hr_dev->caps.srqwqe_hop_num)); -roce_set_field (srq_context->byte_4_srqn_srqst, -SRQC_BYTE_4_SRQ_SHIFT_M ,SRQC_BYTE_4_SRQ_SHIFT_S , -ilog2 (srq->wqe_cnt)); - -roce_set_field (srq_context->byte_4_srqn_srqst,SRQC_BYTE_4_SRQN_M , -SRQC_BYTE_4_SRQN_S , srq->srqn); - -roce_set_field (srq_context->byte_8_limit_wl,SRQC_BYTE_8_SRQ_LIMIT_WL_M , -SRQC_BYTE_8_SRQ_LIMIT_WL_S ,0 ); - -roce_set_field (srq_context->byte_12_xrcd,SRQC_BYTE_12_SRQ_XRCD_M , -SRQC_BYTE_12_SRQ_XRCD_S , xrcd); - - srq_context->wqe_bt_ba =cpu_to_le32 ((u32)(dma_handle_wqe >>3 )); - -roce_set_field (srq_context->byte_24_wqe_bt_ba, -SRQC_BYTE_24_SRQ_WQE_BT_BA_M , -SRQC_BYTE_24_SRQ_WQE_BT_BA_S , - dma_handle_wqe >>35 ); - -roce_set_field (srq_context->byte_28_rqws_pd,SRQC_BYTE_28_PD_M , -SRQC_BYTE_28_PD_S , pdn); -roce_set_field (srq_context->byte_28_rqws_pd,SRQC_BYTE_28_RQWS_M , -SRQC_BYTE_28_RQWS_S , srq->max_gs <=0 ?0 : -fls (srq->max_gs -1 )); - - srq_context->idx_bt_ba =cpu_to_le32 (dma_handle_idx >>3 ); -roce_set_field (srq_context->rsv_idx_bt_ba, -SRQC_BYTE_36_SRQ_IDX_BT_BA_M , -SRQC_BYTE_36_SRQ_IDX_BT_BA_S , - dma_handle_idx >>35 ); - - srq_context->idx_cur_blk_addr = -cpu_to_le32 (to_hr_hw_page_addr (mtts_idx[0 ])); -roce_set_field (srq_context->byte_44_idxbufpgsz_addr, -SRQC_BYTE_44_SRQ_IDX_CUR_BLK_ADDR_M , -SRQC_BYTE_44_SRQ_IDX_CUR_BLK_ADDR_S , -upper_32_bits (to_hr_hw_page_addr (mtts_idx[0 ]))); -roce_set_field (srq_context->byte_44_idxbufpgsz_addr, -SRQC_BYTE_44_SRQ_IDX_HOP_NUM_M , -SRQC_BYTE_44_SRQ_IDX_HOP_NUM_S , - hr_dev->caps.idx_hop_num ==HNS_ROCE_HOP_NUM_0 ?0 : - hr_dev->caps.idx_hop_num); - -roce_set_field ( - srq_context->byte_44_idxbufpgsz_addr, -SRQC_BYTE_44_SRQ_IDX_BA_PG_SZ_M , -SRQC_BYTE_44_SRQ_IDX_BA_PG_SZ_S , -to_hr_hw_page_shift (srq->idx_que.mtr.hem_cfg.ba_pg_shift)); -roce_set_field ( - srq_context->byte_44_idxbufpgsz_addr, -SRQC_BYTE_44_SRQ_IDX_BUF_PG_SZ_M , -SRQC_BYTE_44_SRQ_IDX_BUF_PG_SZ_S , -to_hr_hw_page_shift (srq->idx_que.mtr.hem_cfg.buf_pg_shift)); - - srq_context->idx_nxt_blk_addr = -cpu_to_le32 (to_hr_hw_page_addr (mtts_idx[1 ])); -roce_set_field (srq_context->rsv_idxnxtblkaddr, -SRQC_BYTE_52_SRQ_IDX_NXT_BLK_ADDR_M , -SRQC_BYTE_52_SRQ_IDX_NXT_BLK_ADDR_S , -upper_32_bits (to_hr_hw_page_addr (mtts_idx[1 ]))); -roce_set_field (srq_context->byte_56_xrc_cqn, -SRQC_BYTE_56_SRQ_XRC_CQN_M ,SRQC_BYTE_56_SRQ_XRC_CQN_S , - cqn); -roce_set_field (srq_context->byte_56_xrc_cqn, -SRQC_BYTE_56_SRQ_WQE_BA_PG_SZ_M , -SRQC_BYTE_56_SRQ_WQE_BA_PG_SZ_S , -to_hr_hw_page_shift (srq->buf_mtr.hem_cfg.ba_pg_shift)); -roce_set_field (srq_context->byte_56_xrc_cqn, -SRQC_BYTE_56_SRQ_WQE_BUF_PG_SZ_M , -SRQC_BYTE_56_SRQ_WQE_BUF_PG_SZ_S , -to_hr_hw_page_shift (srq->buf_mtr.hem_cfg.buf_pg_shift)); - -roce_set_bit (srq_context->db_record_addr_record_en, -SRQC_BYTE_60_SRQ_RECORD_EN_S ,0 ); -} - -static int hns_roce_v2_modify_srq (struct ib_srq *ibsrq, -struct ib_srq_attr *srq_attr, -enum ib_srq_attr_mask srq_attr_mask, -struct ib_udata *udata) -{ -struct hns_roce_dev *hr_dev =to_hr_dev (ibsrq->device); -struct hns_roce_srq *srq =to_hr_srq (ibsrq); -struct hns_roce_srq_context *srq_context; -struct hns_roce_srq_context *srqc_mask; -struct hns_roce_cmd_mailbox *mailbox; -int ret; - -if (srq_attr_mask &IB_SRQ_LIMIT ) { -if (srq_attr->srq_limit >= srq->wqe_cnt) -return -EINVAL ; - - mailbox =hns_roce_alloc_cmd_mailbox (hr_dev); -if (IS_ERR (mailbox)) -return PTR_ERR (mailbox); - - srq_context = mailbox->buf; - srqc_mask = (struct hns_roce_srq_context *)mailbox->buf +1 ; - -memset (srqc_mask,0xff ,sizeof (*srqc_mask)); - -roce_set_field (srq_context->byte_8_limit_wl, -SRQC_BYTE_8_SRQ_LIMIT_WL_M , -SRQC_BYTE_8_SRQ_LIMIT_WL_S , srq_attr->srq_limit); -roce_set_field (srqc_mask->byte_8_limit_wl, -SRQC_BYTE_8_SRQ_LIMIT_WL_M , -SRQC_BYTE_8_SRQ_LIMIT_WL_S ,0 ); - - ret =hns_roce_cmd_mbox (hr_dev, mailbox->dma,0 , srq->srqn,0 , -HNS_ROCE_CMD_MODIFY_SRQC , -HNS_ROCE_CMD_TIMEOUT_MSECS ); -hns_roce_free_cmd_mailbox (hr_dev, mailbox); -if (ret) { -ibdev_err (&hr_dev->ib_dev, -"failed to process cmd when modifying SRQ, ret = %d\n" , - ret); -return ret; - } - } - -return 0 ; -} - -static int hns_roce_v2_query_srq (struct ib_srq *ibsrq,struct ib_srq_attr *attr) -{ -struct hns_roce_dev *hr_dev =to_hr_dev (ibsrq->device); -struct hns_roce_srq *srq =to_hr_srq (ibsrq); -struct hns_roce_srq_context *srq_context; -struct hns_roce_cmd_mailbox *mailbox; -int limit_wl; -int ret; - - mailbox =hns_roce_alloc_cmd_mailbox (hr_dev); -if (IS_ERR (mailbox)) -return PTR_ERR (mailbox); - - srq_context = mailbox->buf; - ret =hns_roce_cmd_mbox (hr_dev,0 , mailbox->dma, srq->srqn,0 , -HNS_ROCE_CMD_QUERY_SRQC , -HNS_ROCE_CMD_TIMEOUT_MSECS ); -if (ret) { -ibdev_err (&hr_dev->ib_dev, -"failed to process cmd when querying SRQ, ret = %d\n" , - ret); - goto out; - } - - limit_wl =roce_get_field (srq_context->byte_8_limit_wl, -SRQC_BYTE_8_SRQ_LIMIT_WL_M , -SRQC_BYTE_8_SRQ_LIMIT_WL_S ); - - attr->srq_limit = limit_wl; - attr->max_wr = srq->wqe_cnt -1 ; - attr->max_sge = srq->max_gs; - -memcpy (srq_context, mailbox->buf,sizeof (*srq_context)); - -out: -hns_roce_free_cmd_mailbox (hr_dev, mailbox); -return ret; -} - -static int find_empty_entry (struct hns_roce_idx_que *idx_que, -unsigned long size) -{ -int wqe_idx; - -if (unlikely (bitmap_full (idx_que->bitmap, size))) -return -ENOSPC ; - - wqe_idx =find_first_zero_bit (idx_que->bitmap, size); - -bitmap_set (idx_que->bitmap, wqe_idx,1 ); - -return wqe_idx; -} - -static void fill_idx_queue (struct hns_roce_idx_que *idx_que, -int cur_idx,int wqe_idx) -{ -unsigned int *addr; - - addr = (unsigned int *)hns_roce_buf_offset (idx_que->mtr.kmem, - cur_idx * idx_que->entry_sz); - *addr = wqe_idx; -} - -static int hns_roce_v2_post_srq_recv (struct ib_srq *ibsrq, -const struct ib_recv_wr *wr, -const struct ib_recv_wr **bad_wr) -{ -struct hns_roce_dev *hr_dev =to_hr_dev (ibsrq->device); -struct hns_roce_srq *srq =to_hr_srq (ibsrq); -struct hns_roce_v2_wqe_data_seg *dseg; -struct hns_roce_v2_db srq_db; -unsigned long flags; -int ret =0 ; -int wqe_idx; -void *wqe; -int nreq; -int ind; -int i; - -spin_lock_irqsave (&srq->lock, flags); - - ind = srq->head & (srq->wqe_cnt -1 ); - -for (nreq =0 ; wr; ++nreq, wr = wr->next) { -if (unlikely (wr->num_sge > srq->max_gs)) { - ret = -EINVAL ; - *bad_wr = wr; -break ; - } - -if (unlikely (srq->head == srq->tail)) { - ret = -ENOMEM ; - *bad_wr = wr; -break ; - } - - wqe_idx =find_empty_entry (&srq->idx_que, srq->wqe_cnt); -if (wqe_idx <0 ) { - ret = -ENOMEM ; - *bad_wr = wr; -break ; - } - -fill_idx_queue (&srq->idx_que, ind, wqe_idx); - wqe =get_srq_wqe (srq, wqe_idx); - dseg = (struct hns_roce_v2_wqe_data_seg *)wqe; - -for (i =0 ; i < wr->num_sge; ++i) { - dseg[i].len =cpu_to_le32 (wr->sg_list[i].length); - dseg[i].lkey =cpu_to_le32 (wr->sg_list[i].lkey); - dseg[i].addr =cpu_to_le64 (wr->sg_list[i].addr); - } - -if (i < srq->max_gs) { - dseg[i].len =0 ; - dseg[i].lkey =cpu_to_le32 (0x100 ); - dseg[i].addr =0 ; - } - - srq->wrid[wqe_idx] = wr->wr_id; - ind = (ind +1 ) & (srq->wqe_cnt -1 ); - } - -if (likely (nreq)) { - srq->head += nreq; - -/* -* Make sure that descriptors are written before -* doorbell record. -*/ -wmb (); - - srq_db.byte_4 = -cpu_to_le32 (HNS_ROCE_V2_SRQ_DB <<V2_DB_BYTE_4_CMD_S | - (srq->srqn &V2_DB_BYTE_4_TAG_M )); - srq_db.parameter =cpu_to_le32 (srq->head); - -hns_roce_write64 (hr_dev, (__le32 *)&srq_db, srq->db_reg_l); - - } - -spin_unlock_irqrestore (&srq->lock, flags); - -return ret; -} -static const struct hns_roce_dfx_hw hns_roce_dfx_hw_v2 = { .query_cqc_info = hns_roce_v2_query_cqc_info, };
Signature · 1 line
2.8.1
PATCH v4 5/5 RDMA/hns: Optimize SRQ buffer size calculating process
In reply to Weihang LiFrom: Xi Wang <[email protected]>
Optimize the SRQ's WQE buffer parameters calculating process to make the codes more readable by using new functions about multi-hop addressing to calculating capabilities of SRQ.
--- drivers/infiniband/hw/hns/hns_roce_device.h | 2 +- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 29 ++++++++++++++--------------- drivers/infiniband/hw/hns/hns_roce_srq.c | 16 ++++++++-------- 3 files changed, 23 insertions(+), 24 deletions(-)
drivers/infiniband/hw/hns/hns_roce_device.h +1 -1
@@ -472,7 +472,7 @@ struct hns_roce_cq {struct hns_roce_idx_que {struct hns_roce_mtr mtr; -int entry_sz; +int entry_shift;unsigned long *bitmap; };
drivers/infiniband/hw/hns/hns_roce_hw_v2.c +14 -15
@@ -699,6 +699,12 @@ static void *get_srq_wqe(struct hns_roce_srq *srq, int n)return hns_roce_buf_offset (srq->buf_mtr.kmem, n << srq->wqe_shift); } +static void *get_idx_buf (struct hns_roce_idx_que *idx_que,int n) +{ +return hns_roce_buf_offset (idx_que->mtr.kmem, + n << idx_que->entry_shift); +} +static void hns_roce_free_srq_wqe (struct hns_roce_srq *srq,int wqe_index) {/* always called with interrupts disabled. */
@@ -725,16 +731,6 @@ static int find_empty_entry(struct hns_roce_idx_que *idx_que,return wqe_idx; } -static void fill_idx_queue (struct hns_roce_idx_que *idx_que, -int cur_idx,int wqe_idx) -{ -unsigned int *addr; - - addr = (unsigned int *)hns_roce_buf_offset (idx_que->mtr.kmem, - cur_idx * idx_que->entry_sz); - *addr = wqe_idx; -} -static int hns_roce_v2_post_srq_recv (struct ib_srq *ibsrq,const struct ib_recv_wr *wr,const struct ib_recv_wr **bad_wr)
@@ -744,6 +740,7 @@ static int hns_roce_v2_post_srq_recv(struct ib_srq *ibsrq,struct hns_roce_v2_wqe_data_seg *dseg;struct hns_roce_v2_db srq_db;unsigned long flags; +__le32 *srq_idx;int ret =0 ;int wqe_idx;void *wqe;
@@ -775,7 +772,6 @@ static int hns_roce_v2_post_srq_recv(struct ib_srq *ibsrq,break ; } -fill_idx_queue (&srq->idx_que, ind, wqe_idx); wqe =get_srq_wqe (srq, wqe_idx); dseg = (struct hns_roce_v2_wqe_data_seg *)wqe;
@@ -791,6 +787,9 @@ static int hns_roce_v2_post_srq_recv(struct ib_srq *ibsrq, dseg[i].addr =0 ; } + srq_idx =get_idx_buf (&srq->idx_que, ind); + *srq_idx =cpu_to_le32 (wqe_idx); + srq->wrid[wqe_idx] = wr->wr_id; ind = (ind +1 ) & (srq->wqe_cnt -1 ); }
@@ -4901,8 +4900,8 @@ static void hns_roce_v2_write_srqc(struct hns_roce_dev *hr_dev,roce_set_field (srq_context->byte_4_srqn_srqst,SRQC_BYTE_4_SRQ_WQE_HOP_NUM_M ,SRQC_BYTE_4_SRQ_WQE_HOP_NUM_S , - (hr_dev->caps.srqwqe_hop_num ==HNS_ROCE_HOP_NUM_0 ?0 : - hr_dev->caps.srqwqe_hop_num)); +to_hr_hem_hopnum (hr_dev->caps.srqwqe_hop_num, + srq->wqe_cnt));roce_set_field (srq_context->byte_4_srqn_srqst,SRQC_BYTE_4_SRQ_SHIFT_M ,SRQC_BYTE_4_SRQ_SHIFT_S ,ilog2 (srq->wqe_cnt));
@@ -4944,8 +4943,8 @@ static void hns_roce_v2_write_srqc(struct hns_roce_dev *hr_dev,roce_set_field (srq_context->byte_44_idxbufpgsz_addr,SRQC_BYTE_44_SRQ_IDX_HOP_NUM_M ,SRQC_BYTE_44_SRQ_IDX_HOP_NUM_S , - hr_dev->caps.idx_hop_num ==HNS_ROCE_HOP_NUM_0 ?0 : - hr_dev->caps.idx_hop_num); +to_hr_hem_hopnum (hr_dev->caps.idx_hop_num, + srq->wqe_cnt)); roce_set_field(srq_context->byte_44_idxbufpgsz_addr, SRQC_BYTE_44_SRQ_IDX_BA_PG_SZ_M,
drivers/infiniband/hw/hns/hns_roce_srq.c +8 -8
@@ -181,16 +181,15 @@ static int alloc_srq_buf(struct hns_roce_dev *hr_dev, struct hns_roce_srq *srq, {struct ib_device *ibdev = &hr_dev->ib_dev;struct hns_roce_buf_attr buf_attr = {}; -int sge_size;int err; - sge_size =roundup_pow_of_two (max (HNS_ROCE_SGE_SIZE , -HNS_ROCE_SGE_SIZE * srq->max_gs)); - - srq->wqe_shift =ilog2 (sge_size); + srq->wqe_shift =ilog2 (roundup_pow_of_two (max (HNS_ROCE_SGE_SIZE , +HNS_ROCE_SGE_SIZE * + srq->max_gs))); buf_attr.page_shift = hr_dev->caps.srqwqe_buf_pg_sz +PAGE_ADDR_SHIFT ; - buf_attr.region[0 ].size = srq->wqe_cnt * sge_size; + buf_attr.region[0 ].size =to_hr_hem_entries_size (srq->wqe_cnt, + srq->wqe_shift); buf_attr.region[0 ].hopnum = hr_dev->caps.srqwqe_hop_num; buf_attr.region_count =1 ; buf_attr.fixed_page = true;
@@ -217,10 +216,11 @@ static int alloc_srq_idx(struct hns_roce_dev *hr_dev, struct hns_roce_srq *srq,struct hns_roce_buf_attr buf_attr = {};int err; - srq->idx_que.entry_sz =HNS_ROCE_IDX_QUE_ENTRY_SZ ; + srq->idx_que.entry_shift =ilog2 (HNS_ROCE_IDX_QUE_ENTRY_SZ ); buf_attr.page_shift = hr_dev->caps.idx_buf_pg_sz +PAGE_ADDR_SHIFT ; - buf_attr.region[0 ].size = srq->wqe_cnt *HNS_ROCE_IDX_QUE_ENTRY_SZ ; + buf_attr.region[0 ].size =to_hr_hem_entries_size (srq->wqe_cnt, + srq->idx_que.entry_shift); buf_attr.region[0 ].hopnum = hr_dev->caps.idx_hop_num; buf_attr.region_count =1 ; buf_attr.fixed_page = true;
Signature · 1 line
2.8.1
Jason Gunthorpe
AppliedQuoted 30 lines · Weihang Li
Patch #1 and #2 aim to use MTR interfaces for PBL buffer instead of MTT, and after this, MTT can be removed completely. Patch #3 and #5 refactor buffer size calculation process for WQE and SRQ. #4 can be considered as a preparation for #5, which just moves code of SRQ together to a more suitable place.
This series looks huge, but most of the modification is to replace and remove old interfaces, and patch #4 also contribute a lot. Actually, the original logic is not changed so much.
Changes since v3:
- Fix a sparce warning about a function that should be static reported by kbuild test robot.Changes since v2:
- Just do a rebase to current for-next branch.Changes since v1:
- Remove meaningless judgment of count in some inline functions in #3.
- Add more information into commit messages of #3 and #5.Xi Wang (4):
RDMA/hns: Optimize PBL buffer allocation process RDMA/hns: Remove unused MTT functions RDMA/hns: Optimize WQE buffer size calculating process RDMA/hns: Optimize SRQ buffer size calculating processYixian Liu (1):
RDMA/hns: Move SRQ code to the reasonable place
Quoted 11 lines
drivers/infiniband/hw/hns/hns_roce_alloc.c | 43 - drivers/infiniband/hw/hns/hns_roce_device.h | 110 +-- drivers/infiniband/hw/hns/hns_roce_hem.c | 105 --- drivers/infiniband/hw/hns/hns_roce_hem.h | 6 - drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 45 +- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 943 ++++++++++---------- drivers/infiniband/hw/hns/hns_roce_main.c | 70 +- drivers/infiniband/hw/hns/hns_roce_mr.c | 1247 +++------------------------ drivers/infiniband/hw/hns/hns_roce_qp.c | 313 +++---- drivers/infiniband/hw/hns/hns_roce_srq.c | 16 +- 10 files changed, 767 insertions(+), 2131 deletions(-)
Good diffstat there..
Applied to for-next
Thanks,
Jason