RDMA/hns: Create QP with selected QPN for bank load balance
Subsystems (2)
PATCH v3 1/1 RDMA/hns: Create QP with selected QPN for bank load balance
From: Yangyang Li <[email protected]>
In order to improve performance by balancing the load between different banks of cache, the QPC cache is desigend to choose one of 8 banks according to lower 3 bits of QPN. The hns driver needs to count the number of QP on each bank and then assigns the QP being created to the bank with the minimum load first.
---
Changes since v2:
- Fixed values of min and max when allocating id, and add 'next' to represent the next id.
Changes since v1:
- Use ida instead of bitmap to allocate QPN.
- Remove CQ parts, which will be reorganized in a separate patch.
Link of previous versions:
v2: https://patchwork.kernel.org/project/linux-rdma/patch/[email protected]/
v1: https://patchwork.kernel.org/project/linux-rdma/patch/[email protected]/
drivers/infiniband/hw/hns/hns_roce_device.h | 15 ++++- drivers/infiniband/hw/hns/hns_roce_qp.c | 100 +++++++++++++++++++++++----- 2 files changed, 96 insertions(+), 19 deletions(-)
drivers/infiniband/hw/hns/hns_roce_device.h +13 -2
@@ -117,6 +117,8 @@#define HNS_ROCE_IDX_QUE_ENTRY_SZ 4#define SRQ_DB_REG 0x230 +#define HNS_ROCE_QP_BANK_NUM 8 +/* The chip implementation of the consumer index is calculated * according to twice the actual EQ depth */
@@ -512,13 +514,22 @@ struct hns_roce_uar_table {struct hns_roce_bitmap bitmap; }; +struct hns_roce_bank { +struct ida ida; +u32 inuse;/* Number of IDs allocated */ +u32 min;/* Lowest ID to allocate. */ +u32 max;/* Highest ID to allocate. */ +u32 next;/* Next ID to allocate. */ +}; +struct hns_roce_qp_table { -struct hns_roce_bitmap bitmap;struct hns_roce_hem_table qp_table;struct hns_roce_hem_table irrl_table;struct hns_roce_hem_table trrl_table;struct hns_roce_hem_table sccc_table;struct mutex scc_mutex; +struct hns_roce_bank bank[HNS_ROCE_QP_BANK_NUM ]; +spinlock_t bank_lock; };struct hns_roce_cq_table {
@@ -768,7 +779,7 @@ struct hns_roce_caps {u32 max_rq_sg;u32 max_extend_sg;int num_qps; -int reserved_qps; +u32 reserved_qps;int num_qpc_timer;int num_cqc_timer;int num_srqs;
drivers/infiniband/hw/hns/hns_roce_qp.c +83 -17
148 lines not on this page · raw message
Signature · 1 line
2.8.1
Quoted 11 lines · Weihang Li
From: Yangyang Li <[email protected]>
In order to improve performance by balancing the load between different banks of cache, the QPC cache is desigend to choose one of 8 banks according to lower 3 bits of QPN. The hns driver needs to count the number of QP on each bank and then assigns the QP being created to the bank with the minimum load first.
Signed-off-by Yangyang Li <[email protected]>Signed-off-by Weihang Li <[email protected]>---
Applied to for-next, thanks
Jason