RDMA/providers: Set max_pkey attribute
Subsystems (4)
cover RDMA/providers: Set max_pkey attribute
This patch set makes sure to set the max_pkeys attribute to the providers that aren't setting it or not setting it correctly.
v1: Drop the efa patch and target for-rc.
Kamal Heib (4):
RDMA/siw: Set max_pkeys attribute
RDMA/cxgb4: Set max_pkeys attribute
RDMA/i40iw: Set max_pkeys attribute
RDMA/usnic: Fix reported max_pkeys attribute
drivers/infiniband/hw/cxgb4/provider.c | 1 + drivers/infiniband/hw/i40iw/i40iw_verbs.c | 1 + drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 2 +- drivers/infiniband/sw/siw/siw_verbs.c | 1 + 4 files changed, 4 insertions(+), 1 deletion(-)
Signature · 1 line
2.25.4
PATCH 1/4 RDMA/siw: Set max_pkeys attribute
Make sure to set the max_pkeys attribute to indicate the maximum number of partitions supported by the siw device.
--- drivers/infiniband/sw/siw/siw_verbs.c | 1 + 1 file changed, 1 insertion(+)
drivers/infiniband/sw/siw/siw_verbs.c +1 -0
@@ -151,6 +151,7 @@ int siw_query_device(struct ib_device *base_dev, struct ib_device_attr *attr, attr->max_srq = sdev->attrs.max_srq; attr->max_srq_sge = sdev->attrs.max_srq_sge; attr->max_srq_wr = sdev->attrs.max_srq_wr; + attr->max_pkeys =1 ; attr->page_size_cap =PAGE_SIZE ; attr->vendor_id =SIW_VENDOR_ID ; attr->vendor_part_id = sdev->vendor_part_id;
Signature · 1 line
2.25.4
Re: [PATCH for-rc v1 1/4] RDMA/siw: Set max_pkeys attribute
-----"Kamal Heib" <[email protected]> wrote: -----
Quoted 33 lines
To: [email protected]
From: "Kamal Heib" <[email protected]>
Date: 07/06/2020 11:11AM
Cc: "Jason Gunthorpe" <[email protected]>, "Doug Ledford" <[email protected]>, "Kamal Heib" <[email protected]>, "Bernard Metzler" <[email protected]>
Subject: [EXTERNAL] [PATCH for-rc v1 1/4] RDMA/siw: Setmax_pkeysattributeMake sure to set the
max_pkeysattribute to indicate the maximum number
of partitions supported by the siw device.Fixes 303ae1cdfdf7 ("rdma/siw: application interface")Signed-off-by Kamal Heib <[email protected]>Cc Bernard Metzler <[email protected]>---
drivers/infiniband/sw/siw/siw_verbs.c| 1 +1 file changed, 1 insertion(+)drivers/infiniband/sw/siw/siw_verbs.c +1
-0@@ -151,6 +151,7 @@ int siw_query_device(struct ib_device *base_dev,struct ib_device_attr *attr, attr->max_srq = sdev->attrs.max_srq; attr->max_srq_sge = sdev->attrs.max_srq_sge; attr->max_srq_wr = sdev->attrs.max_srq_wr; + attr->max_pkeys =1 ; attr->page_size_cap =PAGE_SIZE ; attr->vendor_id =SIW_VENDOR_ID ;
Quoted 5 lines
attr->vendor_part_id = sdev->vendor_part_id;Signature · 1 line
2.25.4
Thank you!
PATCH 2/4 RDMA/cxgb4: Set max_pkeys attribute
In reply to Kamal HeibMake sure to set the max_pkeys attribute to indicate the maximum number of partitions supported by the cxgb4 device.
--- drivers/infiniband/hw/cxgb4/provider.c | 1 + 1 file changed, 1 insertion(+)
drivers/infiniband/hw/cxgb4/provider.c +1 -0
@@ -298,6 +298,7 @@ static int c4iw_query_device(struct ib_device *ibdev, struct ib_device_attr *pro props->local_ca_ack_delay =0 ; props->max_fast_reg_page_list_len =t4_max_fr_depth (dev->rdev.lldi.ulptx_memwrite_dsgl && use_dsgl); + props->max_pkeys =1 ;return 0 ; }
Signature · 1 line
2.25.4
PATCH 3/4 RDMA/i40iw: Set max_pkeys attribute
In reply to Kamal HeibMake sure to set the max_pkeys attribute to indicate the maximum number of partitions supported by the i40iw device.
--- drivers/infiniband/hw/i40iw/i40iw_verbs.c | 1 + 1 file changed, 1 insertion(+)
drivers/infiniband/hw/i40iw/i40iw_verbs.c +1 -0
@@ -84,6 +84,7 @@ static int i40iw_query_device(struct ib_device *ibdev, props->max_qp_init_rd_atom = props->max_qp_rd_atom; props->atomic_cap =IB_ATOMIC_NONE ; props->max_fast_reg_page_list_len =I40IW_MAX_PAGES_PER_FMR ; + props->max_pkeys =1 ;return 0 ; }
Signature · 1 line
2.25.4
PATCH 4/4 RDMA/usnic: Fix reported max_pkeys attribute
In reply to Kamal HeibMake sure to report the right max_pkeys attribute value to indicate the maximum number of partitions supported by the usnic device.
--- drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/hw/usnic/usnic_ib_verbs.c +1 -1
@@ -309,7 +309,7 @@ int usnic_ib_query_device(struct ib_device *ibdev, props->max_pd =USNIC_UIOM_MAX_PD_CNT ; props->max_mr =USNIC_UIOM_MAX_MR_CNT ; props->local_ca_ack_delay =0 ; - props->max_pkeys =0 ; + props->max_pkeys =1 ; props->atomic_cap =IB_ATOMIC_NONE ; props->masked_atomic_cap = props->atomic_cap; props->max_qp_rd_atom =0 ;
Signature · 1 line
2.25.4
Jason Gunthorpe
maintainerQuoted 10 lines · Kamal Heib
This patch set makes sure to set the
max_pkeysattribute to the providers that aren't setting it or not setting it correctly.v1: Drop the efa patch and target for-rc.
Kamal Heib (4):
RDMA/siw: Setmax_pkeysattribute
RDMA/cxgb4: Setmax_pkeysattribute
RDMA/i40iw: Setmax_pkeysattribute
RDMA/usnic: Fix reportedmax_pkeysattribute
Why should iwarp have a 1 pkey value?
Jason
Quoted 12 lines · Jason Gunthorpe
Quoted 10 lines · Kamal Heib
This patch set makes sure to set the
max_pkeysattribute to the providers that aren't setting it or not setting it correctly.v1: Drop the efa patch and target for-rc.
Kamal Heib (4):
RDMA/siw: Setmax_pkeysattribute
RDMA/cxgb4: Setmax_pkeysattribute
RDMA/i40iw: Setmax_pkeysattribute
RDMA/usnic: Fix reportedmax_pkeysattribute
Quoted 3 lines
Why should iwarp have a 1 pkey value?
Jason
That is a good question :-)
My Idea in this patchset was to match between the reported pkey_tbl_len and the max_pkeys attribute that the providers expose.
But after taking a deeper look now, I see that the RDMA core requires from all providers to implement the query_pkey() callback, which before
[1] commit that will cause the provider driver not to load. For IB providers the requirement make sense, also for RoCE providers, because there is a requirement by the RoCE Spec to support the default PKey, For iwarp providers, This doesn't make sense and I think that they decided to do the same as RoCE and to avoid the driver load failure.
Probably, The requirement from the RDMA core needs to be changed and the query_pkey() callback needs to be removed from the iwarp providers, Thoughts?
------>8------
git grep -n IB_MANDATORY_FUNC drivers/infiniband/ | grep pkeydrivers/infiniband/core/device.c:275: IB_MANDATORY_FUNC(query_pkey),
------>8------
[1] - 6780c4fa9d6e ("RDMA: Add indication for in kernel API support to IB device")
Thanks,
Kamal
Jason Gunthorpe
maintainerQuoted 30 lines · Kamal Heib
Quoted 12 lines · Jason Gunthorpe
Quoted 10 lines · Kamal Heib
This patch set makes sure to set the
max_pkeysattribute to the providers that aren't setting it or not setting it correctly.v1: Drop the efa patch and target for-rc.
Kamal Heib (4):
RDMA/siw: Setmax_pkeysattribute
RDMA/cxgb4: Setmax_pkeysattribute
RDMA/i40iw: Setmax_pkeysattribute
RDMA/usnic: Fix reportedmax_pkeysattributeQuoted 3 lines
Why should iwarp have a 1 pkey value?
Jason
That is a good question :-)
My Idea in this patchset was to match between the reported
pkey_tbl_lenand themax_pkeysattribute that the providers expose.But after taking a deeper look now, I see that the RDMA core requires from all providers to implement the
query_pkey()callback, which before
[1] commit that will cause the provider driver not to load. For IB providers the requirement make sense, also for RoCE providers, because there is a requirement by the RoCE Spec to support the default PKey, For iwarp providers, This doesn't make sense and I think that they decided to do the same as RoCE and to avoid the driver load failure.
Quoted 3 lines
Probably, The requirement from the RDMA core needs to be changed and the
query_pkey()callback needs to be removed from the iwarp providers, Thoughts?
Sure
But then the pkey table size is 0 right?
Jason
Quoted 31 lines · Jason Gunthorpe
Quoted 30 lines · Kamal Heib
Quoted 12 lines · Jason Gunthorpe
Quoted 10 lines · Kamal Heib
This patch set makes sure to set the
max_pkeysattribute to the providers that aren't setting it or not setting it correctly.v1: Drop the efa patch and target for-rc.
Kamal Heib (4):
RDMA/siw: Setmax_pkeysattribute
RDMA/cxgb4: Setmax_pkeysattribute
RDMA/i40iw: Setmax_pkeysattribute
RDMA/usnic: Fix reportedmax_pkeysattributeQuoted 3 lines
Why should iwarp have a 1 pkey value?
Jason
That is a good question :-)
My Idea in this patchset was to match between the reported
pkey_tbl_lenand themax_pkeysattribute that the providers expose.But after taking a deeper look now, I see that the RDMA core requires from all providers to implement the
query_pkey()callback, which before
[1] commit that will cause the provider driver not to load. For IB providers the requirement make sense, also for RoCE providers, because there is a requirement by the RoCE Spec to support the default PKey, For iwarp providers, This doesn't make sense and I think that they decided to do the same as RoCE and to avoid the driver load failure.
Quoted 8 lines
Quoted 3 lines
Probably, The requirement from the RDMA core needs to be changed and the
query_pkey()callback needs to be removed from the iwarp providers, Thoughts?Sure
But then the pkey table size is 0 right?
Correct, also there are required changes in the RDMA core:
1- Avoid exposing the pkeys sysfs entries for the iwarp providers. 2- Avoid allocating the pkey cache for the iwarp providers.
I've started this work and I hope to post patches in the upcoming days.
Thanks,
Kamal
Quoted 1 line
Jason