Skip to contentSkip to conversation

USNIC and RAW enhancements series

PATCH In review Leon Romanovsky 5 participants 11 messages Jan 24, 2017 – Feb 14, 2017 4 messages not in the archive

Subsystems (12)
InfiniBand Mellanox MLX5 IB driver Mellanox MLX4 IB driver Soft-RoCE driver (rxe) +8 more subsystems
lore
Share & tools
lore

Leon Romanovsky

author
#0 lore raw

cover USNIC and RAW enhancements series

Hi Doug,

Please find below second version og USNIC and RAW protocols enhancements from Or.

from: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

This series adds the USNIC and RAW protocols to the core and relevant devices (usnic, and mlx5/mlx4 over Eth ports) such that now the protocol part of the caps covers all in-tree HW/SW devices.

It also contains a cleanup patch to the HW/SW drivers such that they use ib_query_port when needed to query their own ports and not apply their internal callback. The patch doesn't add new functionality but makes the different drivers to be behave the same in that respect and makes the IB stack code a little bit more robust and clear.

Thanks
---
Changelog v0->v1:
* Change memset calls to use actual size of filled variable and not structure.
---
CC: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
CC: Parav Pandit <parav-VPRAkNaXOzVWk0Htik3J/[email protected]>
CC: Steve Wise <[email protected]>
CC: Dennis Dalessandro <[email protected]>
CC: Lijun Ou <[email protected]>
CC: Faisal Latif <[email protected]>
CC: Shiraz Saleem <[email protected]>
CC: Selvin Xavier <selvin.xavier-1wcpHE2jlwO1Z/[email protected]>
CC: Christian Benvenuti <[email protected]>
CC: Dave Goodell <[email protected]>
CC: Adit Ranadive <[email protected]>
CC: Michal Kalderon <Michal.Kalderon-YGCgFSpz5w/[email protected]>
CC: Rajesh Borundia <rajesh.borundia-YGCgFSpz5w/[email protected]>
---
This patchset was generated against commit
f502d834950a ("net/mlx5: Activate support for 4K UARs") and it applies cleanly on k.o/for-4.11 branch.

Available in the "topic/raw-qp-query-enhance" topic branch of this git repo:
git://git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git

Or for browsing:
https://git.kernel.org/cgit/linux/kernel/git/leon/linux-rdma.git/log/?h=topic/raw-qp-query-enhance

Or Gerlitz (5):
IB/core: Add raw packet protocol
IB/mlx5: Support raw packet protocol
IB/mlx4: Support raw packet protocol
IB: Add protocol for USNIC
IB: Query port through the core instead of directly calling the driver
handler

 drivers/infiniband/hw/cxgb3/iwch_provider.c     |  7 ++++---
 drivers/infiniband/hw/cxgb4/provider.c          |  8 ++++----
 drivers/infiniband/hw/hfi1/verbs.c              |  1 +
 drivers/infiniband/hw/hns/hns_roce_main.c       |  7 ++++---
 drivers/infiniband/hw/i40iw/i40iw_verbs.c       |  8 ++++----
 drivers/infiniband/hw/mlx4/alias_GUID.c         |  1 +
 drivers/infiniband/hw/mlx4/main.c               | 23 +++++++++++++----------
 drivers/infiniband/hw/mlx4/sysfs.c              |  1 +
 drivers/infiniband/hw/mlx5/mad.c                |  2 +-
 drivers/infiniband/hw/mlx5/main.c               | 17 +++++++++++------
 drivers/infiniband/hw/mthca/mthca_provider.c    |  9 +++++----
 drivers/infiniband/hw/nes/nes_verbs.c           |  5 +++--
 drivers/infiniband/hw/ocrdma/ocrdma_main.c      |  9 +++++----
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c     |  1 +
 drivers/infiniband/hw/qedr/verbs.c              |  9 +++++----
 drivers/infiniband/hw/qib/qib_verbs.c           |  1 +
 drivers/infiniband/hw/usnic/usnic_ib_main.c     |  4 +++-
 drivers/infiniband/hw/usnic/usnic_ib_verbs.c    |  2 +-
 drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c  |  5 +++--
 drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c |  4 ++--
 drivers/infiniband/sw/rdmavt/vt.c               |  7 ++++---
 drivers/infiniband/sw/rxe/rxe_verbs.c           |  6 ++++--
 include/rdma/ib_verbs.h                         | 16 ++++++++++++++++
 23 files changed, 97 insertions(+), 56 deletions(-)

--
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20170124110239.31555-1-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org

author
#2 lore raw

PATCH 1/5 IB/core: Add raw packet protocol

From: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

Define raw packet protocol which comes to denote this port supports working with raw ethernet frames, e.g as done with RAW_PACKET QPs.

Signed-off-by Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 include/rdma/ib_verbs.h | 8 ++++++++
 1 file changed, 8 insertions(+)
include/rdma/ib_verbs.h +8 -0
index 958a24d8fae7..68fff341d3b6 100644
@@ -485,6 +485,7 @@ static inline struct rdma_hw_stats *rdma_alloc_hw_stats_struct(
 #define RDMA_CORE_CAP_PROT_ROCE         0x00200000
 #define RDMA_CORE_CAP_PROT_IWARP        0x00400000
 #define RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP 0x00800000
+#define RDMA_CORE_CAP_PROT_RAW_PACKET   0x01000000
 
 #define RDMA_CORE_PORT_IBA_IB          (RDMA_CORE_CAP_PROT_IB  \
 					| RDMA_CORE_CAP_IB_MAD \

@@ -508,6 +509,8 @@ static inline struct rdma_hw_stats *rdma_alloc_hw_stats_struct(
 #define RDMA_CORE_PORT_INTEL_OPA       (RDMA_CORE_PORT_IBA_IB  \
 					| RDMA_CORE_CAP_OPA_MAD)
 
+#define RDMA_CORE_PORT_RAW_PACKET	(RDMA_CORE_CAP_PROT_RAW_PACKET)
+
 struct ib_port_attr {
 	u64			subnet_prefix;
 	enum ib_port_state	state;

@@ -2307,6 +2310,11 @@ static inline bool rdma_ib_or_roce(const struct ib_device *device, u8 port_num)
 		rdma_protocol_roce(device, port_num);
 }
 
+static inline bool rdma_protocol_raw_packet(const struct ib_device *device, u8 port_num)
+{
+	return device->port_immutable[port_num].core_cap_flags & RDMA_CORE_CAP_PROT_RAW_PACKET;
+}
+
 /**
  * rdma_cap_ib_mad - Check if the port of a device supports Infiniband
  * Management Datagrams.
Signature · 6 lines
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20170124110239.31555-2-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org

Reviewed-by
#4 lore raw

RE: [PATCH rdma-next V1 1/5] IB/core: Add raw packet protocol

Quoted 10 lines

Subject: [PATCH rdma-next V1 1/5] IB/core: Add raw packet protocol

From: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

Define raw packet protocol which comes to denote this port supports working with raw ethernet frames, e.g as done with RAW_PACKET QPs.

Signed-off-by Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>

Core changes look fine.

Reviewed-by Steve Wise <[email protected]>

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#5 lore raw

PATCH 2/5 IB/mlx5: Support raw packet protocol

In reply to a message not in the archive

From: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

Mark support for the new raw packet protocol on Eth ports.

Signed-off-by Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/hw/mlx5/main.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/infiniband/hw/mlx5/main.c +4 -2
index a191b9327b0c..4e8e78f5149f 100644
@@ -2858,11 +2858,13 @@ static u32 get_core_cap_flags(struct ib_device *ibdev)
 	if (ll == IB_LINK_LAYER_INFINIBAND)
 		return RDMA_CORE_PORT_IBA_IB;
 
+	ret = RDMA_CORE_PORT_RAW_PACKET;
+
 	if (!(l3_type_cap & MLX5_ROCE_L3_TYPE_IPV4_CAP))
-		return 0;
+		return ret;
 
 	if (!(l3_type_cap & MLX5_ROCE_L3_TYPE_IPV6_CAP))
-		return 0;
+		return ret;
 
 	if (roce_version_cap & MLX5_ROCE_VERSION_1_CAP)
 		ret |= RDMA_CORE_PORT_IBA_ROCE;
Signature · 6 lines
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#6 lore raw

PATCH 3/5 IB/mlx4: Support raw packet protocol

In reply to a message not in the archive

From: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

Mark support for the new raw packet protocol on Eth ports.

Signed-off-by Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/hw/mlx4/main.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/infiniband/hw/mlx4/main.c +5 -2
index c8413fc120e6..7019b9450408 100644
@@ -2536,16 +2536,19 @@ static int mlx4_port_immutable(struct ib_device *ibdev, u8 port_num,
 
 	if (mlx4_ib_port_link_layer(ibdev, port_num) == IB_LINK_LAYER_INFINIBAND) {
 		immutable->core_cap_flags = RDMA_CORE_PORT_IBA_IB;
+		immutable->max_mad_size = IB_MGMT_MAD_SIZE;
 	} else {
 		if (mdev->dev->caps.flags & MLX4_DEV_CAP_FLAG_IBOE)
 			immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE;
 		if (mdev->dev->caps.flags2 & MLX4_DEV_CAP_FLAG2_ROCE_V1_V2)
 			immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE |
 				RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;
+		immutable->core_cap_flags |= RDMA_CORE_PORT_RAW_PACKET;
+		if (immutable->core_cap_flags & (RDMA_CORE_PORT_IBA_ROCE |
+		    RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP))
+			immutable->max_mad_size = IB_MGMT_MAD_SIZE;
 	}
 
-	immutable->max_mad_size = IB_MGMT_MAD_SIZE;
-
 	return 0;
 }
Signature · 6 lines
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#7 lore raw

PATCH 4/5 IB: Add protocol for USNIC

In reply to a message not in the archive

From: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

Add protocol definition for the proprietary the USNIC driver.

Signed-off-by Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/hw/usnic/usnic_ib_main.c | 1 +
 include/rdma/ib_verbs.h                     | 8 ++++++++
 2 files changed, 9 insertions(+)
drivers/infiniband/hw/usnic/usnic_ib_main.c +1 -0
index 0a89a955550b..dde0b23f6559 100644
@@ -325,6 +325,7 @@ static int usnic_port_immutable(struct ib_device *ibdev, u8 port_num,
 	if (err)
 		return err;
 
+	immutable->core_cap_flags = RDMA_CORE_PORT_USNIC;
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
include/rdma/ib_verbs.h +8 -0
index 68fff341d3b6..d5109e63fa7e 100644
@@ -486,6 +486,7 @@ static inline struct rdma_hw_stats *rdma_alloc_hw_stats_struct(
 #define RDMA_CORE_CAP_PROT_IWARP        0x00400000
 #define RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP 0x00800000
 #define RDMA_CORE_CAP_PROT_RAW_PACKET   0x01000000
+#define RDMA_CORE_CAP_PROT_USNIC        0x02000000
 
 #define RDMA_CORE_PORT_IBA_IB          (RDMA_CORE_CAP_PROT_IB  \
 					| RDMA_CORE_CAP_IB_MAD \

@@ -511,6 +512,8 @@ static inline struct rdma_hw_stats *rdma_alloc_hw_stats_struct(
 
 #define RDMA_CORE_PORT_RAW_PACKET	(RDMA_CORE_CAP_PROT_RAW_PACKET)
 
+#define RDMA_CORE_PORT_USNIC		(RDMA_CORE_CAP_PROT_USNIC)
+
 struct ib_port_attr {
 	u64			subnet_prefix;
 	enum ib_port_state	state;

@@ -2315,6 +2318,11 @@ static inline bool rdma_protocol_raw_packet(const struct ib_device *device, u8 p
 	return device->port_immutable[port_num].core_cap_flags & RDMA_CORE_CAP_PROT_RAW_PACKET;
 }
 
+static inline bool rdma_protocol_usnic(const struct ib_device *device, u8 port_num)
+{
+	return device->port_immutable[port_num].core_cap_flags & RDMA_CORE_CAP_PROT_USNIC;
+}
+
 /**
  * rdma_cap_ib_mad - Check if the port of a device supports Infiniband
  * Management Datagrams.
Signature · 6 lines
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20170124110239.31555-5-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org

Reviewed-by
#9 lore raw

RE: [PATCH rdma-next V1 4/5] IB: Add protocol for USNIC

Quoted 10 lines

-----Original Message-----
From: Leon Romanovsky [mailto:[email protected]]
Sent: Tuesday, January 24, 2017 3:03 AM
To: [email protected]
Cc: [email protected]; Or Gerlitz; Parav Pandit; Steve Wise; Dennis Dalessandro; Lijun Ou; Faisal Latif; Shiraz Saleem; Selvin Xavier; Christian Benvenuti (benve); Dave Goodell (dgoodell); Adit Ranadive; Michal Kalderon;
Rajesh Borundia
Subject: [PATCH rdma-next V1 4/5] IB: Add protocol for USNIC

Quoted 7 lines

From: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

Add protocol definition for the proprietary the USNIC driver.

Signed-off-by Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
Reviewed-by Christian Benvenuti <[email protected]>

/Chris

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#10 lore raw

PATCH 5/5 IB: Query port through the core instead of directly calling the driver handler

In reply to a message not in the archive

From: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

Change the drivers to call ib_query_port in their get port immutable handler instead of their own query port handler.

Doing this required to set the core cap flags of this device before the ib_query_port call is made, since the IB core might need these caps to serve the port query.

Drivers are ensured by the IB core that the port attributes passed to the port query verb implementation are zero, and hence we removed the zeroing from the drivers.

This patch doesn't add any new functionality.

Signed-off-by Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/hw/cxgb3/iwch_provider.c     |  7 ++++---
 drivers/infiniband/hw/cxgb4/provider.c          |  8 ++++----
 drivers/infiniband/hw/hfi1/verbs.c              |  1 +
 drivers/infiniband/hw/hns/hns_roce_main.c       |  7 ++++---
 drivers/infiniband/hw/i40iw/i40iw_verbs.c       |  8 ++++----
 drivers/infiniband/hw/mlx4/alias_GUID.c         |  1 +
 drivers/infiniband/hw/mlx4/main.c               | 18 +++++++++---------
 drivers/infiniband/hw/mlx4/sysfs.c              |  1 +
 drivers/infiniband/hw/mlx5/mad.c                |  2 +-
 drivers/infiniband/hw/mlx5/main.c               | 11 +++++++----
 drivers/infiniband/hw/mthca/mthca_provider.c    |  9 +++++----
 drivers/infiniband/hw/nes/nes_verbs.c           |  5 +++--
 drivers/infiniband/hw/ocrdma/ocrdma_main.c      |  9 +++++----
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c     |  1 +
 drivers/infiniband/hw/qedr/verbs.c              |  9 +++++----
 drivers/infiniband/hw/qib/qib_verbs.c           |  1 +
 drivers/infiniband/hw/usnic/usnic_ib_main.c     |  5 +++--
 drivers/infiniband/hw/usnic/usnic_ib_verbs.c    |  2 +-
 drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c  |  5 +++--
 drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c |  4 ++--
 drivers/infiniband/sw/rdmavt/vt.c               |  7 ++++---
 drivers/infiniband/sw/rxe/rxe_verbs.c           |  6 ++++--
 22 files changed, 73 insertions(+), 54 deletions(-)
drivers/infiniband/hw/cxgb3/iwch_provider.c +4 -3
index 9d5fe1853da4..9e39252a570a 100644
@@ -1133,7 +1133,7 @@ static int iwch_query_port(struct ib_device *ibdev,
 	dev = to_iwch_dev(ibdev);
 	netdev = dev->rdev.port_info.lldevs[port-1];
 
-	memset(props, 0, sizeof(struct ib_port_attr));
+	/* props being zeroed by the caller, avoid zeroing it here */
 	props->max_mtu = IB_MTU_4096;
 	if (netdev->mtu >= 4096)
 		props->active_mtu = IB_MTU_4096;

@@ -1338,13 +1338,14 @@ static int iwch_port_immutable(struct ib_device *ibdev, u8 port_num,
 	struct ib_port_attr attr;
 	int err;
 
-	err = iwch_query_port(ibdev, port_num, &attr);
+	immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
+
+	err = ib_query_port(ibdev, port_num, &attr);
 	if (err)
 		return err;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
-	immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
 
 	return 0;
 }
drivers/infiniband/hw/cxgb4/provider.c +4 -4
index 49b51b7e0fd7..5b00b50a484f 100644
@@ -358,8 +358,7 @@ static int c4iw_query_port(struct ib_device *ibdev, u8 port,
 
 	dev = to_c4iw_dev(ibdev);
 	netdev = dev->rdev.lldi.ports[port-1];
-
-	memset(props, 0, sizeof(struct ib_port_attr));
+	/* props being zeroed by the caller, avoid zeroing it here */
 	props->max_mtu = IB_MTU_4096;
 	if (netdev->mtu >= 4096)
 		props->active_mtu = IB_MTU_4096;

@@ -505,13 +504,14 @@ static int c4iw_port_immutable(struct ib_device *ibdev, u8 port_num,
 	struct ib_port_attr attr;
 	int err;
 
-	err = c4iw_query_port(ibdev, port_num, &attr);
+	immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
+
+	err = ib_query_port(ibdev, port_num, &attr);
 	if (err)
 		return err;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
-	immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
 
 	return 0;
 }
drivers/infiniband/hw/hfi1/verbs.c +1 -0
index 95ed4d6da510..f2d4e0428725 100644
@@ -1384,6 +1384,7 @@ static int query_port(struct rvt_dev_info *rdi, u8 port_num,
 	struct hfi1_pportdata *ppd = &dd->pport[port_num - 1];
 	u16 lid = ppd->lid;
 
+	/* props being zeroed by the caller, avoid zeroing it here */
 	props->lid = lid ? lid : 0;
 	props->lmc = ppd->lmc;
 	/* OPA logical states match IB logical states */
drivers/infiniband/hw/hns/hns_roce_main.c +4 -3
index 4953d9cb83a7..7be8158f15ee 100644
@@ -249,7 +249,7 @@ static int hns_roce_query_port(struct ib_device *ib_dev, u8 port_num,
 	assert(port_num > 0);
 	port = port_num - 1;
 
-	memset(props, 0, sizeof(*props));
+	/* props being zeroed by the caller, avoid zeroing it here */
 
 	props->max_mtu = hr_dev->caps.max_mtu;
 	props->gid_tbl_len = hr_dev->caps.gid_table_len[port];

@@ -400,14 +400,15 @@ static int hns_roce_port_immutable(struct ib_device *ib_dev, u8 port_num,
 	struct ib_port_attr attr;
 	int ret;
 
-	ret = hns_roce_query_port(ib_dev, port_num, &attr);
+	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE;
+
+	ret = ib_query_port(ib_dev, port_num, &attr);
 	if (ret)
 		return ret;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
 
-	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE;
 	immutable->max_mad_size = IB_MGMT_MAD_SIZE;
 
 	return 0;
drivers/infiniband/hw/i40iw/i40iw_verbs.c +4 -4
index 29e97df9e1a7..3e830486417b 100644
@@ -97,8 +97,7 @@ static int i40iw_query_port(struct ib_device *ibdev,
 	struct i40iw_device *iwdev = to_iwdev(ibdev);
 	struct net_device *netdev = iwdev->netdev;
 
-	memset(props, 0, sizeof(*props));
-
+	/* props being zeroed by the caller, avoid zeroing it here */
 	props->max_mtu = IB_MTU_4096;
 	if (netdev->mtu >= 4096)
 		props->active_mtu = IB_MTU_4096;

@@ -2506,14 +2505,15 @@ static int i40iw_port_immutable(struct ib_device *ibdev, u8 port_num,
 	struct ib_port_attr attr;
 	int err;
 
-	err = i40iw_query_port(ibdev, port_num, &attr);
+	immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
+
+	err = ib_query_port(ibdev, port_num, &attr);
 
 	if (err)
 		return err;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
-	immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
 
 	return 0;
 }
drivers/infiniband/hw/mlx4/alias_GUID.c +1 -0
index 06020c54db20..ea24230ea0d4 100644
@@ -499,6 +499,7 @@ static int set_guid_rec(struct ib_device *ibdev,
 	struct list_head *head =
 		&dev->sriov.alias_guid.ports_guid[port - 1].cb_list;
 
+	memset(&attr, 0, sizeof(attr));
 	err = __mlx4_ib_query_port(ibdev, port, &attr, 1);
 	if (err) {
 		pr_debug("mlx4_ib_query_port failed (err: %d), port: %d\n",
drivers/infiniband/hw/mlx4/main.c +9 -9
index 7019b9450408..646dcbffcf27 100644
@@ -741,7 +741,7 @@ int __mlx4_ib_query_port(struct ib_device *ibdev, u8 port,
 {
 	int err;
 
-	memset(props, 0, sizeof *props);
+	/* props being zeroed by the caller, avoid zeroing it here */
 
 	err = mlx4_ib_port_link_layer(ibdev, port) == IB_LINK_LAYER_INFINIBAND ?
 		ib_link_query_port(ibdev, port, props, netw_view) :

@@ -1014,7 +1014,7 @@ static int mlx4_ib_modify_port(struct ib_device *ibdev, u8 port, int mask,
 
 	mutex_lock(&mdev->cap_mask_mutex);
 
-	err = mlx4_ib_query_port(ibdev, port, &attr);
+	err = ib_query_port(ibdev, port, &attr);
 	if (err)
 		goto out;
 

@@ -2527,13 +2527,6 @@ static int mlx4_port_immutable(struct ib_device *ibdev, u8 port_num,
 	struct mlx4_ib_dev *mdev = to_mdev(ibdev);
 	int err;
 
-	err = mlx4_ib_query_port(ibdev, port_num, &attr);
-	if (err)
-		return err;
-
-	immutable->pkey_tbl_len = attr.pkey_tbl_len;
-	immutable->gid_tbl_len = attr.gid_tbl_len;
-
 	if (mlx4_ib_port_link_layer(ibdev, port_num) == IB_LINK_LAYER_INFINIBAND) {
 		immutable->core_cap_flags = RDMA_CORE_PORT_IBA_IB;
 		immutable->max_mad_size = IB_MGMT_MAD_SIZE;

@@ -2549,6 +2542,13 @@ static int mlx4_port_immutable(struct ib_device *ibdev, u8 port_num,
 			immutable->max_mad_size = IB_MGMT_MAD_SIZE;
 	}
 
+	err = ib_query_port(ibdev, port_num, &attr);
+	if (err)
+		return err;
+
+	immutable->pkey_tbl_len = attr.pkey_tbl_len;
+	immutable->gid_tbl_len = attr.gid_tbl_len;
+
 	return 0;
 }
drivers/infiniband/hw/mlx4/sysfs.c +1 -0
index 69fb5ba94d0f..0ba5ba7540c8 100644
@@ -226,6 +226,7 @@ static int add_port_entries(struct mlx4_ib_dev *device, int port_num)
 	int ret = 0 ;
 	struct ib_port_attr attr;
 
+	memset(&attr, 0, sizeof(attr));
 	/* get the physical gid and pkey table sizes.*/
 	ret = __mlx4_ib_query_port(&device->ib_dev, port_num, &attr, 1);
 	if (ret)
drivers/infiniband/hw/mlx5/mad.c +1 -1
index 39e58489dcc2..4e58b8fce29c 100644
@@ -515,7 +515,7 @@ int mlx5_query_mad_ifc_port(struct ib_device *ibdev, u8 port,
 	if (!in_mad || !out_mad)
 		goto out;
 
-	memset(props, 0, sizeof(*props));
+	/* props being zeroed by the caller, avoid zeroing it here */
 
 	init_query_mad(in_mad);
 	in_mad->attr_id  = IB_SMP_ATTR_PORT_INFO;
drivers/infiniband/hw/mlx5/main.c +7 -4
index 4e8e78f5149f..b8511b9aae9d 100644
@@ -174,7 +174,7 @@ static int mlx5_query_port_roce(struct ib_device *device, u8 port_num,
 	enum ib_mtu ndev_ib_mtu;
 	u16 qkey_viol_cntr;
 
-	memset(props, 0, sizeof(*props));
+	/* props being zeroed by the caller, avoid zeroing it here */
 
 	props->port_cap_flags  |= IB_PORT_CM_SUP;
 	props->port_cap_flags  |= IB_PORT_IP_BASED_GIDS;

@@ -830,7 +830,7 @@ static int mlx5_query_hca_port(struct ib_device *ibdev, u8 port,
 		goto out;
 	}
 
-	memset(props, 0, sizeof(*props));
+	/* props being zeroed by the caller, avoid zeroing it here */
 
 	err = mlx5_query_hca_vport_context(mdev, 0, port, 0, rep);
 	if (err)

@@ -978,7 +978,7 @@ static int mlx5_ib_modify_port(struct ib_device *ibdev, u8 port, int mask,
 
 	mutex_lock(&dev->cap_mask_mutex);
 
-	err = mlx5_ib_query_port(ibdev, port, &attr);
+	err = ib_query_port(ibdev, port, &attr);
 	if (err)
 		goto out;
 

@@ -2564,6 +2564,7 @@ static int get_port_caps(struct mlx5_ib_dev *dev)
 	}
 
 	for (port = 1; port <= MLX5_CAP_GEN(dev->mdev, num_ports); port++) {
+		memset(pprops, 0, sizeof(*pprops));
 		err = mlx5_ib_query_port(&dev->ib_dev, port, pprops);
 		if (err) {
 			mlx5_ib_warn(dev, "query_port %d failed %d\n",

@@ -2883,7 +2884,9 @@ static int mlx5_port_immutable(struct ib_device *ibdev, u8 port_num,
 	enum rdma_link_layer ll = mlx5_ib_port_link_layer(ibdev, port_num);
 	int err;
 
-	err = mlx5_ib_query_port(ibdev, port_num, &attr);
+	immutable->core_cap_flags = get_core_cap_flags(ibdev);
+
+	err = ib_query_port(ibdev, port_num, &attr);
 	if (err)
 		return err;
drivers/infiniband/hw/mthca/mthca_provider.c +5 -4
index d31708742ba5..ce163184e742 100644
@@ -146,7 +146,7 @@ static int mthca_query_port(struct ib_device *ibdev,
 	if (!in_mad || !out_mad)
 		goto out;
 
-	memset(props, 0, sizeof *props);
+	/* props being zeroed by the caller, avoid zeroing it here */
 
 	init_query_mad(in_mad);
 	in_mad->attr_id  = IB_SMP_ATTR_PORT_INFO;

@@ -212,7 +212,7 @@ static int mthca_modify_port(struct ib_device *ibdev,
 	if (mutex_lock_interruptible(&to_mdev(ibdev)->cap_mask_mutex))
 		return -ERESTARTSYS;
 
-	err = mthca_query_port(ibdev, port, &attr);
+	err = ib_query_port(ibdev, port, &attr);
 	if (err)
 		goto out;
 

@@ -1166,13 +1166,14 @@ static int mthca_port_immutable(struct ib_device *ibdev, u8 port_num,
 	struct ib_port_attr attr;
 	int err;
 
-	err = mthca_query_port(ibdev, port_num, &attr);
+	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_IB;
+
+	err = ib_query_port(ibdev, port_num, &attr);
 	if (err)
 		return err;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
-	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_IB;
 	immutable->max_mad_size = IB_MGMT_MAD_SIZE;
 
 	return 0;
drivers/infiniband/hw/nes/nes_verbs.c +3 -2
index aff9fb14768b..b7179f4ac3a6 100644
@@ -475,7 +475,7 @@ static int nes_query_port(struct ib_device *ibdev, u8 port, struct ib_port_attr
 	struct nes_vnic *nesvnic = to_nesvnic(ibdev);
 	struct net_device *netdev = nesvnic->netdev;
 
-	memset(props, 0, sizeof(*props));
+	/* props being zeroed by the caller, avoid zeroing it here */
 
 	props->max_mtu = IB_MTU_4096;
 

@@ -3670,13 +3670,14 @@ static int nes_port_immutable(struct ib_device *ibdev, u8 port_num,
 	struct ib_port_attr attr;
 	int err;
 
+	immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
+
 	err = nes_query_port(ibdev, port_num, &attr);
 	if (err)
 		return err;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
-	immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
 
 	return 0;
 }
drivers/infiniband/hw/ocrdma/ocrdma_main.c +5 -4
index 896071502739..3e43bdc81e7a 100644
@@ -93,15 +93,16 @@ static int ocrdma_port_immutable(struct ib_device *ibdev, u8 port_num,
 	int err;
 
 	dev = get_ocrdma_dev(ibdev);
-	err = ocrdma_query_port(ibdev, port_num, &attr);
+	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE;
+	if (ocrdma_is_udp_encap_supported(dev))
+		immutable->core_cap_flags |= RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP;
+
+	err = ib_query_port(ibdev, port_num, &attr);
 	if (err)
 		return err;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
-	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE;
-	if (ocrdma_is_udp_encap_supported(dev))
-		immutable->core_cap_flags |= RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP;
 	immutable->max_mad_size = IB_MGMT_MAD_SIZE;
 
 	return 0;
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c +1 -0
index 6af44f8db3d5..013d15c7e593 100644
@@ -210,6 +210,7 @@ int ocrdma_query_port(struct ib_device *ibdev,
 	struct ocrdma_dev *dev;
 	struct net_device *netdev;
 
+	/* props being zeroed by the caller, avoid zeroing it here */
 	dev = get_ocrdma_dev(ibdev);
 	if (port > 1) {
 		pr_err("%s(%d) invalid_port=0x%x\n", __func__,
drivers/infiniband/hw/qedr/verbs.c +5 -4
index 57c8de208077..91a16d38d1fc 100644
@@ -238,8 +238,8 @@ int qedr_query_port(struct ib_device *ibdev, u8 port, struct ib_port_attr *attr)
 	}
 
 	rdma_port = dev->ops->rdma_query_port(dev->rdma_ctx);
-	memset(attr, 0, sizeof(*attr));
 
+	/* *attr being zeroed by the caller, avoid zeroing it here */
 	if (rdma_port->port_state == QED_RDMA_PORT_UP) {
 		attr->state = IB_PORT_ACTIVE;
 		attr->phys_state = 5;

@@ -3549,14 +3549,15 @@ int qedr_port_immutable(struct ib_device *ibdev, u8 port_num,
 	struct ib_port_attr attr;
 	int err;
 
-	err = qedr_query_port(ibdev, port_num, &attr);
+	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE |
+				    RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;
+
+	err = ib_query_port(ibdev, port_num, &attr);
 	if (err)
 		return err;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
-	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE |
-				    RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;
 	immutable->max_mad_size = IB_MGMT_MAD_SIZE;
 
 	return 0;
drivers/infiniband/hw/qib/qib_verbs.c +1 -0
index 4b54c0ddd08a..9161574601f6 100644
@@ -1303,6 +1303,7 @@ static int qib_query_port(struct rvt_dev_info *rdi, u8 port_num,
 	enum ib_mtu mtu;
 	u16 lid = ppd->lid;
 
+	/* props being zeroed by the caller, avoid zeroing it here */
 	props->lid = lid ? lid : be16_to_cpu(IB_LID_PERMISSIVE);
 	props->lmc = ppd->lmc;
 	props->state = dd->f_iblink_state(ppd->lastibcstat);
drivers/infiniband/hw/usnic/usnic_ib_main.c +3 -2
index dde0b23f6559..4f5a45db08e1 100644
@@ -321,11 +321,12 @@ static int usnic_port_immutable(struct ib_device *ibdev, u8 port_num,
 	struct ib_port_attr attr;
 	int err;
 
-	err = usnic_ib_query_port(ibdev, port_num, &attr);
+	immutable->core_cap_flags = RDMA_CORE_PORT_USNIC;
+
+	err = ib_query_port(ibdev, port_num, &attr);
 	if (err)
 		return err;
 
-	immutable->core_cap_flags = RDMA_CORE_PORT_USNIC;
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
drivers/infiniband/hw/usnic/usnic_ib_verbs.c +1 -1
index 74819a7951e2..0ba274ff7be6 100644
@@ -330,7 +330,7 @@ int usnic_ib_query_port(struct ib_device *ibdev, u8 port,
 
 	mutex_lock(&us_ibdev->usdev_lock);
 	__ethtool_get_link_ksettings(us_ibdev->netdev, &cmd);
-	memset(props, 0, sizeof(*props));
+	/* props being zeroed by the caller, avoid zeroing it here */
 
 	props->lid = 0;
 	props->lmc = 1;
drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c +3 -2
index 231a1ce1f4be..b8b0081de478 100644
@@ -132,13 +132,14 @@ static int pvrdma_port_immutable(struct ib_device *ibdev, u8 port_num,
 	struct ib_port_attr attr;
 	int err;
 
-	err = pvrdma_query_port(ibdev, port_num, &attr);
+	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE;
+
+	err = ib_query_port(ibdev, port_num, &attr);
 	if (err)
 		return err;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
-	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE;
 	immutable->max_mad_size = IB_MGMT_MAD_SIZE;
 	return 0;
 }
drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c +2 -2
index 54891370d18a..b3a982be8006 100644
@@ -135,7 +135,7 @@ int pvrdma_query_port(struct ib_device *ibdev, u8 port,
 		return err;
 	}
 
-	memset(props, 0, sizeof(*props));
+	/* props being zeroed by the caller, avoid zeroing it here */
 
 	props->state = pvrdma_port_state_to_ib(resp->attrs.state);
 	props->max_mtu = pvrdma_mtu_to_ib(resp->attrs.max_mtu);

@@ -275,7 +275,7 @@ int pvrdma_modify_port(struct ib_device *ibdev, u8 port, int mask,
 	}
 
 	mutex_lock(&vdev->port_mutex);
-	ret = pvrdma_query_port(ibdev, port, &attr);
+	ret = ib_query_port(ibdev, port, &attr);
 	if (ret)
 		goto out;
drivers/infiniband/sw/rdmavt/vt.c +4 -3
index d430c2f7cec4..1165639a914b 100644
@@ -165,7 +165,7 @@ static int rvt_query_port(struct ib_device *ibdev, u8 port_num,
 		return -EINVAL;
 
 	rvp = rdi->ports[port_index];
-	memset(props, 0, sizeof(*props));
+	/* props being zeroed by the caller, avoid zeroing it here */
 	props->sm_lid = rvp->sm_lid;
 	props->sm_sl = rvp->sm_sl;
 	props->port_cap_flags = rvp->port_cap_flags;

@@ -326,13 +326,14 @@ static int rvt_get_port_immutable(struct ib_device *ibdev, u8 port_num,
 	if (port_index < 0)
 		return -EINVAL;
 
-	err = rvt_query_port(ibdev, port_num, &attr);
+	immutable->core_cap_flags = rdi->dparms.core_cap_flags;
+
+	err = ib_query_port(ibdev, port_num, &attr);
 	if (err)
 		return err;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
-	immutable->core_cap_flags = rdi->dparms.core_cap_flags;
 	immutable->max_mad_size = rdi->dparms.max_mad_size;
 
 	return 0;
drivers/infiniband/sw/rxe/rxe_verbs.c +4 -2
index beb7021ff18a..371ef3bac8d4 100644
@@ -86,6 +86,7 @@ static int rxe_query_port(struct ib_device *dev,
 
 	port = &rxe->port;
 
+	/* *attr being zeroed by the caller, avoid zeroing it here */
 	*attr = port->attr;
 
 	mutex_lock(&rxe->usdev_lock);

@@ -261,13 +262,14 @@ static int rxe_port_immutable(struct ib_device *dev, u8 port_num,
 	int err;
 	struct ib_port_attr attr;
 
-	err = rxe_query_port(dev, port_num, &attr);
+	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;
+
+	err = ib_query_port(dev, port_num, &attr);
 	if (err)
 		return err;
 
 	immutable->pkey_tbl_len = attr.pkey_tbl_len;
 	immutable->gid_tbl_len = attr.gid_tbl_len;
-	immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;
 	immutable->max_mad_size = IB_MGMT_MAD_SIZE;
 
 	return 0;
Signature · 6 lines
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20170124110239.31555-6-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org

Reviewed-by
#12 lore raw

RE: [PATCH rdma-next V1 5/5] IB: Query port through the core instead of directly calling the driver handler

Quoted 16 lines

Subject: [PATCH rdma-next V1 5/5] IB: Query port through the core instead of directly calling the driver handler

From: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

Change the drivers to call ib_query_port in their get port immutable handler instead of their own query port handler.

Doing this required to set the core cap flags of this device before the ib_query_port call is made, since the IB core might need these caps to serve the port query.

Drivers are ensured by the IB core that the port attributes passed to the port query verb implementation are zero, and hence we removed the zeroing from the drivers.

Quoted 5 lines

This patch doesn't add any new functionality.

Signed-off-by Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>

cxgb* parts look fine.

Reviewed-by Steve Wise <[email protected]>

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Acked-by
#13 lore raw

Re: [PATCH rdma-next V1 5/5] IB: Query port through the core instead of directly calling the driver handler

In reply to a message not in the archive
Quoted 13 lines · Leon Romanovsky

From: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

Change the drivers to call ib_query_port in their get port immutable handler instead of their own query port handler.

Doing this required to set the core cap flags of this device before the ib_query_port call is made, since the IB core might need these caps to serve the port query.

Drivers are ensured by the IB core that the port attributes passed to the port query verb implementation are zero, and hence we removed the zeroing from the drivers.

Quoted 5 lines

This patch doesn't add any new functionality.

Signed-off-by Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>

vmw_pvrdma changes look fine.

Acked-by Adit Ranadive <[email protected]>

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#14 lore raw
In reply to a message not in the archive
Quoted 21 lines · Leon Romanovsky

Hi Doug,

Please find below second version og USNIC and RAW protocols enhancements from Or.

from: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>

This series adds the USNIC and RAW protocols to the core and relevant devices
(usnic, and mlx5/mlx4 over Eth ports) such that now the protocol part of the
caps covers all in-tree HW/SW devices.

It also contains a cleanup patch to the HW/SW drivers such that they use
ib_query_port when needed to query their own ports and not apply their
internal callback. The patch doesn't add new functionality but makes the different drivers to be behave the same in that respect and makes the IB stack code a little bit more robust and clear.

Quoted 1 line

Thanks

Thanks, series applied.

Signature · 4 lines
Doug Ledford <[email protected]>
    GPG KeyID: B826A3330E572FDD
   
Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD