RDMA/hns: Support inline data in extented sge space for RC
Subsystems (2)
PATCH 1/1 RDMA/hns: Support inline data in extented sge space for RC
HIP08 supports RC inline up to size of 32 Bytes, and all data should be put into SQWQE. For HIP09, this capability is extended to 1024 Bytes, if length of data is longer than 32 Bytes, they will be filled into extended sge space.
Signed-off-by Weihang Li <[email protected]>
--- drivers/infiniband/hw/hns/hns_roce_device.h | 2 + drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 196 +++++++++++++++++++++------- drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 3 + drivers/infiniband/hw/hns/hns_roce_qp.c | 13 +- 4 files changed, 162 insertions(+), 52 deletions(-)
drivers/infiniband/hw/hns/hns_roce_device.h +2 -0
@@ -656,6 +656,8 @@ struct hns_roce_qp {struct hns_roce_sge sge;u32 next_sge; +enum ib_mtu path_mtu; +u32 max_inline_data;/* 0: flush needed, 1: unneeded */ unsigned long flush_flag;
drivers/infiniband/hw/hns/hns_roce_hw_v2.c +150 -46
235 lines not on this page · raw message
drivers/infiniband/hw/hns/hns_roce_hw_v2.h +3 -0
@@ -60,6 +60,7 @@#define HNS_ROCE_V2_MAX_SQ_SGE_NUM 64#define HNS_ROCE_V2_MAX_EXTEND_SGE_NUM 0x200000#define HNS_ROCE_V2_MAX_SQ_INLINE 0x20 +#define HNS_ROCE_V2_MAX_RC_INL_INN_SZ 32#define HNS_ROCE_V2_UAR_NUM 256#define HNS_ROCE_V2_PHY_UAR_NUM 1#define HNS_ROCE_V2_MAX_IRQ_NUM 65
@@ -1187,6 +1188,8 @@ struct hns_roce_v2_rc_send_wqe {#define V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S 0#define V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M GENMASK(23, 0) +#define V2_RC_SEND_WQE_BYTE_20_INL_TYPE_S 31 +struct hns_roce_wqe_frmr_seg {__le32 pbl_size;__le32 mode_buf_pg_sz;
drivers/infiniband/hw/hns/hns_roce_qp.c +7 -6
@@ -551,10 +551,9 @@ static int set_kernel_sq_size(struct hns_roce_dev *hr_dev,int ret;if (!cap->max_send_wr || cap->max_send_wr > hr_dev->caps.max_wqes || - cap->max_send_sge > hr_dev->caps.max_sq_sg || - cap->max_inline_data > hr_dev->caps.max_sq_inline) { + cap->max_send_sge > hr_dev->caps.max_sq_sg) {ibdev_err (ibdev, -"failed to check SQ WR, SGE or inline num, ret = %d.\n" , +"failed to check SQ WR or SGE num, ret = %d.\n" , -EINVAL );return -EINVAL ; }
@@ -577,9 +576,6 @@ static int set_kernel_sq_size(struct hns_roce_dev *hr_dev, cap->max_send_wr = cnt; cap->max_send_sge = hr_qp->sq.max_gs; -/* We don't support inline sends for kernel QPs (yet) */ - cap->max_inline_data =0 ; -return 0 ; }
@@ -847,6 +843,11 @@ static int set_qp_param(struct hns_roce_dev *hr_dev, struct hns_roce_qp *hr_qp, hr_qp->ibqp.qp_type = init_attr->qp_type; +if (init_attr->cap.max_inline_data > hr_dev->caps.max_sq_inline) + init_attr->cap.max_inline_data = hr_dev->caps.max_sq_inline; + + hr_qp->max_inline_data = init_attr->cap.max_inline_data; +if (init_attr->sq_sig_type ==IB_SIGNAL_ALL_WR ) hr_qp->sq_signal_bits =IB_SIGNAL_ALL_WR ;else
Signature · 1 line
2.8.1
Quoted 5 lines · Weihang Li
HIP08 supports RC inline up to size of 32 Bytes, and all data should be put into SQWQE. For HIP09, this capability is extended to 1024 Bytes, if length of data is longer than 32 Bytes, they will be filled into extended sge space.
Quoted 7 lines
Signed-off-by Weihang Li <[email protected]>--- drivers/infiniband/hw/hns/hns_roce_device.h | 2 + drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 196 +++++++++++++++++++++------- drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 3 + drivers/infiniband/hw/hns/hns_roce_qp.c | 13 +- 4 files changed, 162 insertions(+), 52 deletions(-)
Applied to for-next
Thanks,
Jason