RDMA: modify_port improvements
Subsystems (4)
v2 cover RDMA: modify_port improvements
Changelog:
v2: Include fixes lines.
This series includes three patches which fix the return values from modify_port() callbacks when they aren't supported.
Kamal Heib (4):
RDMA/core: Fix return code when modify_port isn't supported RDMA/hns: Remove unsupported modify_port callback RDMA/ocrdma: Remove unsupported modify_port callback RDMA/qedr: Remove unsupported modify_port callback
drivers/infiniband/core/device.c | 2 +- drivers/infiniband/hw/hns/hns_roce_main.c | 7 ------- drivers/infiniband/hw/ocrdma/ocrdma_main.c | 1 - drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 6 ------ drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 2 -- drivers/infiniband/hw/qedr/main.c | 1 - drivers/infiniband/hw/qedr/verbs.c | 6 ------ drivers/infiniband/hw/qedr/verbs.h | 2 -- 8 files changed, 1 insertion(+), 26 deletions(-)
Signature · 1 line
2.20.1
PATCH v2 1/4 RDMA/core: Fix return code when modify_port isn't supported
The proper return code is "-EOPNOTSUPP" when modify_port callback is not supported.
--- drivers/infiniband/core/device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/core/device.c +1 -1
@@ -2397,7 +2397,7 @@ int ib_modify_port(struct ib_device *device, port_modify_mask, port_modify); else - rc =rdma_protocol_roce (device, port_num) ?0 : -ENOSYS ; + rc =rdma_protocol_roce (device, port_num) ?0 : -EOPNOTSUPP ;return rc; }EXPORT_SYMBOL (ib_modify_port);
Signature · 1 line
2.20.1
Michal Kalderon
maintainerRE: [EXT] [PATCH for-next v2 1/4] RDMA/core: Fix return code when modify_port isn't supported
Quoted 21 lines
From: Kamal Heib <[email protected]>
Sent: Wednesday, October 16, 2019 10:23 AMExternal Email
----------------------------------------------------------------------
The proper return code is "-EOPNOTSUPP" whenmodify_portcallback is not supported.Fixes 61e0962d5221 ("IB: Avoid ib_modify_port() failure for RoCE devices")Signed-off-by Kamal Heib <[email protected]>--- drivers/infiniband/core/device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/infiniband/core/device.c +0
-0
drivers/infiniband/core/device.c int ib_modify_port(struct ib_device *device, · line 2401
Quoted 6 lines
drivers/infiniband/core/device.c +1
-1@@ -2397,7 +2397,7 @@ int ib_modify_port(struct ib_device *device, port_modify_mask, port_modify); else - rc =rdma_protocol_roce (device, port_num) ?0 : -ENOSYS ; + rc =rdma_protocol_roce (device, port_num) ?0 : -EOPNOTSUPP ;
This is a bit confusing, looks like for RoCE it's ok not to have a callback but for the The other protocols it's required. For iWARP for example there also isn't a modify-port. Is there any other protocol except ib that this is relevant to ?
If not perhaps modify rdma_protocol_roce(..)? to rdma_protocol_ib(...)? -EOPNOTSUPP : 0?
Quoted 5 lines
drivers/infiniband/core/device.c +0
-0return rc; }
EXPORT_SYMBOL(ib_modify_port);
--
2.20.1
Kamal Heib
Reviewed-by Leon RomanovskyRe: [EXT] [PATCH for-next v2 1/4] RDMA/core: Fix return code when modify_port isn't supported
Quoted 24 lines · Michal Kalderon
Quoted 24 lines
From: Kamal Heib <[email protected]>
Sent: Wednesday, October 16, 2019 10:23 AMExternal Email
----------------------------------------------------------------------
The proper return code is "-EOPNOTSUPP" whenmodify_portcallback is not supported.Fixes 61e0962d5221 ("IB: Avoid ib_modify_port() failure for RoCE devices")Signed-off-by Kamal Heib <[email protected]>--- drivers/infiniband/core/device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/infiniband/core/device.c +0
-0@@ -2397,7 +2397,7 @@ int ib_modify_port(struct ib_device *device, port_modify_mask, port_modify); else
Quoted 9 lines
drivers/infiniband/core/device.c int ib_modify_port(struct ib_device *device, · line 2401
Quoted 3 lines
drivers/infiniband/core/device.c +1
-1- rc =rdma_protocol_roce (device, port_num) ?0 : -ENOSYS ; + rc =rdma_protocol_roce (device, port_num) ?0 : -EOPNOTSUPP ;This is a bit confusing, looks like for RoCE it's ok not to have a callback but for the The other protocols it's required. For iWARP for example there also isn't a modify-port. Is there any other protocol except ib that this is relevant to ?
If not perhaps modifyrdma_protocol_roce(..)? tordma_protocol_ib(...)?-EOPNOTSUPP: 0?
Yes, I agree this is confusing.
This change was introduced by the following commit to avoid the failures of ib_modify_port() calls from CM when the protocol is RoCE, I also see that almost all providers that support RoCE return success from the modify_port() callback (hns, mlx4, mlx5, ocrdma, qedr), except rxe and vmw_pvrdma which I think they shouldn't.
So, I suggest adding a check to CM avoid calling ib_modify_port() when the protocol is RoCE and cleanup the mess from the providers, thoughts?
commit 61e0962d52216f2e5bab59bb055f1210e41f484f
Author: Selvin Xavier <[email protected]> Date: Wed Aug 23 01:08:07 2017 -0700
IB: Avoid ib_modify_port() failure for RoCE devices
IB CM calls ib_modify_port() irrespective of link layer. If the failure is returned, the mad agent gets unregistered for those devices. Recently, modify_port() hook was removed from some of the low level drivers as it was always returning success. This breaks rdma connection establishment over those devices.
For ethernet devices, Qkey violation and port capabilities are not applicable. So returning success for RoCE when modify_port hook is is not implemented.
drivers/infiniband/core/device.c +7 -4
@@ -1005,14 +1005,17 @@ int ib_modify_port(struct ib_device *device, u8 port_num, int port_modify_mask, struct ib_port_modify *port_modify) { - if (!device->modify_port) - return -ENOSYS; + int rc;if (!rdma_is_port_valid (device, port_num))return -EINVAL ; -return device->modify_port (device, port_num, port_modify_mask, - port_modify); +if (device->modify_port) + rc = device->modify_port (device, port_num, port_modify_mask, + port_modify); +else + rc =rdma_protocol_roce (device, port_num) ?0 : -ENOSYS ; +return rc; }EXPORT_SYMBOL (ib_modify_port);
Quoted 8 lines
Quoted 7 lines
drivers/infiniband/core/device.c +0
-0return rc; }
EXPORT_SYMBOL(ib_modify_port);
--
2.20.1
Michal Kalderon
maintainerRE: [EXT] [PATCH for-next v2 1/4] RDMA/core: Fix return code when modify_port isn't supported
Quoted 43 lines
From: [email protected] <linux-rdma- [email protected]> On Behalf Of Kamal Heib
Quoted 16 lines · Michal Kalderon
Quoted 16 lines
From: Kamal Heib <[email protected]>
Sent: Wednesday, October 16, 2019 10:23 AMExternal Email
--------------------------------------------------------------------
-- The proper return code is "-EOPNOTSUPP" whenmodify_portcallback is not supported.Fixes 61e0962d5221 ("IB: Avoid ib_modify_port() failure for RoCEdevices")
Signed-off-by Kamal Heib <[email protected]>--- drivers/infiniband/core/device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 15 lines
drivers/infiniband/core/device.c int ib_modify_port(struct ib_device *device, · line 2401
Quoted 12 lines
drivers/infiniband/core/device.c +1
-1@@ -2397,7 +2397,7 @@ int ib_modify_port(struct ib_device *device, port_modify_mask, port_modify); else - rc =rdma_protocol_roce (device, port_num) ?0 : -ENOSYS ; + rc =rdma_protocol_roce (device, port_num) ?0 : -EOPNOTSUPP ;This is a bit confusing, looks like for RoCE it's ok not to have a callback but for the The other protocols it's required. For iWARP for
example there also isn't a modify-port.
Quoted 2 lines
Is there any other protocol except ib that this is relevant to ? If not perhaps modify
rdma_protocol_roce(..)? tordma_protocol_ib(...)? -EOPNOTSUPP : 0?
Quoted 1 line
Yes, I agree this is confusing.
Quoted 8 lines
This change was introduced by the following commit to avoid the failures of
ib_modify_port()calls from CM when the protocol is RoCE, I also see that almost all providers that support RoCE return success from themodify_port()callback (hns, mlx4, mlx5, ocrdma, qedr), except rxe andvmw_pvrdmawhich I think they shouldn't.So, I suggest adding a check to CM avoid calling
ib_modify_port()when the protocol is RoCE and cleanup the mess from the providers, thoughts?
I think we can leave the logic inside the function ib_modify_port, and just return Success if the protocol isn't IB.
Quoted 56 lines
commit
61e0962d52216f2e5bab59bb055f1210e41f484fAuthor: Selvin Xavier <[email protected]> Date: Wed Aug 23 01:08:07 2017 -0700IB: Avoid
ib_modify_port()failure for RoCE devicesIB CM calls
ib_modify_port()irrespective of link layer. If the failure is returned, the mad agent gets unregistered for those devices. Recently,modify_port()hook was removed from some of the low level drivers as it was always returning success. This breaks rdma connection establishment over those devices.
For ethernet devices, Qkey violation and port capabilities are not applicable. So returning success for RoCE whenmodify_porthook is is not implemented.Cc Leon Romanovsky <[email protected]>Signed-off-by Selvin Xavier <[email protected]>Reviewed-by Leon Romanovsky <[email protected]>Signed-off-by Doug Ledford <[email protected]>drivers/infiniband/core/device.c +7
-4@@ -1005,14 +1005,17 @@ int ib_modify_port(struct ib_device *device, u8 port_num, int port_modify_mask, struct ib_port_modify *port_modify) { - if (!device->modify_port) - return -ENOSYS; + int rc;if (!rdma_is_port_valid (device, port_num))return -EINVAL ; -return device->modify_port (device, port_num, port_modify_mask, - port_modify); +if (device->modify_port) + rc = device->modify_port (device, port_num, port_modify_mask, + port_modify); +else + rc =rdma_protocol_roce (device, port_num) ?0 : -ENOSYS ; +return rc; }EXPORT_SYMBOL (ib_modify_port);Quoted 8 lines
Quoted 7 lines
drivers/infiniband/core/device.c +0
-0return rc; }
EXPORT_SYMBOL(ib_modify_port);
--
2.20.1
Re: [EXT] [PATCH for-next v2 1/4] RDMA/core: Fix return code when modify_port isn't supported
Quoted 49 lines · Michal Kalderon
Quoted 49 lines
From: [email protected] <linux-rdma- [email protected]> On Behalf Of Kamal Heib
Quoted 16 lines · Michal Kalderon
Quoted 16 lines
From: Kamal Heib <[email protected]>
Sent: Wednesday, October 16, 2019 10:23 AMExternal Email
--------------------------------------------------------------------
-- The proper return code is "-EOPNOTSUPP" whenmodify_portcallback is not supported.Fixes 61e0962d5221 ("IB: Avoid ib_modify_port() failure for RoCEdevices")
Signed-off-by Kamal Heib <[email protected]>--- drivers/infiniband/core/device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 15 lines
drivers/infiniband/core/device.c int ib_modify_port(struct ib_device *device, · line 2401
Quoted 12 lines
drivers/infiniband/core/device.c +1
-1@@ -2397,7 +2397,7 @@ int ib_modify_port(struct ib_device *device, port_modify_mask, port_modify); else - rc =rdma_protocol_roce (device, port_num) ?0 : -ENOSYS ; + rc =rdma_protocol_roce (device, port_num) ?0 : -EOPNOTSUPP ;This is a bit confusing, looks like for RoCE it's ok not to have a callback but for the The other protocols it's required. For iWARP for
example there also isn't a modify-port.
Quoted 2 lines
Is there any other protocol except ib that this is relevant to ? If not perhaps modify
rdma_protocol_roce(..)? tordma_protocol_ib(...)? -EOPNOTSUPP : 0?
Quoted 1 line
Yes, I agree this is confusing.
This change was introduced by the following commit to avoid the failures of
ib_modify_port()calls from CM when the protocol is RoCE, I also see that almost all providers that support RoCE return success from themodify_port()callback (hns, mlx4, mlx5, ocrdma, qedr), except rxe andvmw_pvrdmawhich I think they shouldn't.
Quoted 5 lines
Quoted 2 lines
So, I suggest adding a check to CM avoid calling
ib_modify_port()when the protocol is RoCE and cleanup the mess from the providers, thoughts?I think we can leave the logic inside the function
ib_modify_port, and just return Success if the protocol isn't IB.
OK, I'll fix it in v3.
Quoted 56 lines
Quoted 56 lines
commit
61e0962d52216f2e5bab59bb055f1210e41f484fAuthor: Selvin Xavier <[email protected]> Date: Wed Aug 23 01:08:07 2017 -0700IB: Avoid
ib_modify_port()failure for RoCE devicesIB CM calls
ib_modify_port()irrespective of link layer. If the failure is returned, the mad agent gets unregistered for those devices. Recently,modify_port()hook was removed from some of the low level drivers as it was always returning success. This breaks rdma connection establishment over those devices.
For ethernet devices, Qkey violation and port capabilities are not applicable. So returning success for RoCE whenmodify_porthook is is not implemented.Cc Leon Romanovsky <[email protected]>Signed-off-by Selvin Xavier <[email protected]>Reviewed-by Leon Romanovsky <[email protected]>Signed-off-by Doug Ledford <[email protected]>drivers/infiniband/core/device.c +7
-4@@ -1005,14 +1005,17 @@ int ib_modify_port(struct ib_device *device, u8 port_num, int port_modify_mask, struct ib_port_modify *port_modify) { - if (!device->modify_port) - return -ENOSYS; + int rc;if (!rdma_is_port_valid (device, port_num))return -EINVAL ; -return device->modify_port (device, port_num, port_modify_mask, - port_modify); +if (device->modify_port) + rc = device->modify_port (device, port_num, port_modify_mask, + port_modify); +else + rc =rdma_protocol_roce (device, port_num) ?0 : -ENOSYS ; +return rc; }EXPORT_SYMBOL (ib_modify_port);Quoted 8 lines
Quoted 7 lines
drivers/infiniband/core/device.c +0
-0return rc; }
EXPORT_SYMBOL(ib_modify_port);
--
2.20.1
PATCH v2 2/4 RDMA/hns: Remove unsupported modify_port callback
In reply to Kamal HeibThere is no need to return always zero for function which is not supported.
--- drivers/infiniband/hw/hns/hns_roce_main.c | 7 ------- 1 file changed, 7 deletions(-)
drivers/infiniband/hw/hns/hns_roce_main.c +0 -7
@@ -301,12 +301,6 @@ static int hns_roce_modify_device(struct ib_device *ib_dev, int mask,return 0 ; } -static int hns_roce_modify_port (struct ib_device *ib_dev,u8 port_num,int mask, -struct ib_port_modify *props) -{ -return 0 ; -} -static int hns_roce_alloc_ucontext (struct ib_ucontext *uctx,struct ib_udata *udata) {
@@ -438,7 +432,6 @@ static const struct ib_device_ops hns_roce_dev_ops = { .get_port_immutable = hns_roce_port_immutable, .mmap = hns_roce_mmap, .modify_device = hns_roce_modify_device, - .modify_port = hns_roce_modify_port, .modify_qp = hns_roce_modify_qp, .query_ah = hns_roce_query_ah, .query_device = hns_roce_query_device,
Signature · 1 line
2.20.1
PATCH v2 3/4 RDMA/ocrdma: Remove unsupported modify_port callback
In reply to Kamal HeibThere is no need to return always zero for function which is not supported.
--- drivers/infiniband/hw/ocrdma/ocrdma_main.c | 1 - drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 6 ------ drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 2 -- 3 files changed, 9 deletions(-)
drivers/infiniband/hw/ocrdma/ocrdma_main.c +0 -1
@@ -166,7 +166,6 @@ static const struct ib_device_ops ocrdma_dev_ops = { .get_port_immutable = ocrdma_port_immutable, .map_mr_sg = ocrdma_map_mr_sg, .mmap = ocrdma_mmap, - .modify_port = ocrdma_modify_port, .modify_qp = ocrdma_modify_qp, .poll_cq = ocrdma_poll_cq, .post_recv = ocrdma_post_recv,
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c +0 -6
@@ -190,12 +190,6 @@ int ocrdma_query_port(struct ib_device *ibdev,return 0 ; } -int ocrdma_modify_port (struct ib_device *ibdev,u8 port,int mask, -struct ib_port_modify *props) -{ -return 0 ; -} -static int ocrdma_add_mmap (struct ocrdma_ucontext *uctx,u64 phy_addr,unsigned long len) {
drivers/infiniband/hw/ocrdma/ocrdma_verbs.h +0 -2
@@ -54,8 +54,6 @@ int ocrdma_arm_cq(struct ib_cq *, enum ib_cq_notify_flags flags);int ocrdma_query_device (struct ib_device *,struct ib_device_attr *props,struct ib_udata *uhw);int ocrdma_query_port (struct ib_device *,u8 port,struct ib_port_attr *props); -int ocrdma_modify_port (struct ib_device *,u8 port,int mask, -struct ib_port_modify *props);enum rdma_protocol_type ocrdma_query_protocol (struct ib_device *device,u8 port_num);
Signature · 1 line
2.20.1
PATCH v2 4/4 RDMA/qedr: Remove unsupported modify_port callback
In reply to Kamal HeibThere is no need to return always zero for function which is not supported.
--- drivers/infiniband/hw/qedr/main.c | 1 - drivers/infiniband/hw/qedr/verbs.c | 6 ------ drivers/infiniband/hw/qedr/verbs.h | 2 -- 3 files changed, 9 deletions(-)
drivers/infiniband/hw/qedr/main.c +0 -1
@@ -212,7 +212,6 @@ static const struct ib_device_ops qedr_dev_ops = { .get_link_layer = qedr_link_layer, .map_mr_sg = qedr_map_mr_sg, .mmap = qedr_mmap, - .modify_port = qedr_modify_port, .modify_qp = qedr_modify_qp, .modify_srq = qedr_modify_srq, .poll_cq = qedr_poll_cq,
drivers/infiniband/hw/qedr/verbs.c +0 -6
@@ -250,12 +250,6 @@ int qedr_query_port(struct ib_device *ibdev, u8 port, struct ib_port_attr *attr)return 0 ; } -int qedr_modify_port (struct ib_device *ibdev,u8 port,int mask, -struct ib_port_modify *props) -{ -return 0 ; -} -static int qedr_add_mmap (struct qedr_ucontext *uctx,u64 phy_addr,unsigned long len) {
drivers/infiniband/hw/qedr/verbs.h +0 -2
@@ -35,8 +35,6 @@int qedr_query_device (struct ib_device *ibdev,struct ib_device_attr *attr,struct ib_udata *udata);int qedr_query_port (struct ib_device *,u8 port,struct ib_port_attr *props); -int qedr_modify_port (struct ib_device *,u8 port,int mask, -struct ib_port_modify *props);int qedr_iw_query_gid (struct ib_device *ibdev,u8 port,int index,union ib_gid *gid);
Signature · 1 line
2.20.1