RDMA core and mlx5 fixes and refactoring for 4.15

1) Introduce and use rdma_create_user_ah

Currently there is inconsistency in resolving destination mac address in ib core and provider driver as below.
1. Provider drivers are using different API to resolve dmac.
2. Provider drivers are resolving destination mac address for user space consumers, but don't have to resolve for kernel consumers.

This leads ah_attr->roce.dmac being input field for kernel consumers and treated as output field for user space consumers.

Without proper API, umad module miss resolving destination mac address while creating address handle.

Additionally mlx4 driver resolves incorrect mac address for v4 mapped v6 destination multicast gid.

This patchset allows uverbs modules to invoke ib_core API that resolves destination mac address for user consumers.

This patchset avoids above issues by ib_core resolving destination mac address for roce for user as well as kernel consumers.
It uses ib_resolve_eth_dmac() API which resolves destination mac address for unicast, multicast, v4 mapped v6 addresses, and also link local gid. This eliminates code duplication in provider drivers and also brings uniformity by always resolving dmac in ib_core for all type of GIDs.

This allows to eliminate exporting low level API ib_resolve_eth_dmac(). So its removed as export symbol until a good use case arise.

2) Three simple fixes for IB/core.
3) Patch for mlx5 posted to the mailing list.

Changelog:
v0->v1: Added forgotten patch "IB/core: Introduce and use rdma_create_user_ah"

The patches are available in the git repository at:
git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git tags/rdma-next-2017-10-16

Thanks
---------------------------------------

Jérémy Lefaure (1):
IB/mlx5: Use ARRAY_SIZE

Parav Pandit (5):
IB/core: Introduce and use rdma_create_user_ah
IB: Let ib_core resolve destination mac address IB/core: Fix unable to change lifespan entry for hw_counters IB/core: Fix use workqueue without WQ_MEM_RECLAIM IB/core: Take into account optional UDR, XRC headers and mandatory
ICRC for RoCE MTU

 drivers/infiniband/core/addr.c           |  2 +-
 drivers/infiniband/core/sysfs.c          | 16 ++++++++++-
 drivers/infiniband/core/user_mad.c       |  2 +-
 drivers/infiniband/core/uverbs_cmd.c     | 10 +------
 drivers/infiniband/core/verbs.c          | 48 ++++++++++++++++++++++++++++----
 drivers/infiniband/hw/bnxt_re/ib_verbs.c |  8 ------
 drivers/infiniband/hw/hns/hns_roce_ah.c  | 14 +---------
 drivers/infiniband/hw/mlx4/ah.c          |  8 ++----
 drivers/infiniband/hw/mlx5/ah.c          |  4 ---
 drivers/infiniband/hw/mlx5/odp.c         |  6 ++--
 drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 15 ----------
 include/rdma/ib_addr.h                   |  7 +++--
 include/rdma/ib_pack.h                   | 19 +++++++------
 include/rdma/ib_verbs.h                  | 17 +++++++++--
 14 files changed, 98 insertions(+), 78 deletions(-)

--
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Parav Pandit <parav-VPRAkNaXOzVWk0Htik3J/[email protected]>

Introduce rdma_create_user_ah API which allows passing udata to provider driver and additionally which resolves DMAC for RoCE.

ib_resolve_eth_dmac() resolves destination mac address for unicast, multicast, link local ipv4 mapped ipv6 and ipv6 destination gid entry. This allows all RoCE provider drivers to avoid duplicating such code.

Such change brings consistency where IB core always resolves dmac and pass it to RoCE provider drivers for user and kernel consumers, with this ah_attr->roce.dmac is always an input field for provider drivers.

This uniformity avoids exporting ib_resolve_eth_dmac symbol to providers or other modules. Therefore its removed as exported symbol at later in the patch series.

Now uverbs and umad both makes use of rdma_create_user_ah API which fixes the issue where umad has invalid DMAC for address.

---
 drivers/infiniband/core/user_mad.c   |  2 +-
 drivers/infiniband/core/uverbs_cmd.c | 10 +--------
 drivers/infiniband/core/verbs.c      | 40 ++++++++++++++++++++++++++++++++++--
 include/rdma/ib_verbs.h              | 15 ++++++++++++++
 4 files changed, 55 insertions(+), 12 deletions(-)

index 603acaf91828..4b64dd02e090 100644
@@ -515,7 +515,7 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf,
 		rdma_ah_set_dgid_raw(&ah_attr, packet->mad.hdr.gid);
 	}
 
-	ah = rdma_create_ah(agent->qp->pd, &ah_attr);
+	ah = rdma_create_user_ah(agent->qp->pd, &ah_attr, NULL);
 	if (IS_ERR(ah)) {
 		ret = PTR_ERR(ah);
 		goto err_up;

index 537e5f2cff69..d31e4bc58e9a 100644
@@ -2521,7 +2521,6 @@ ssize_t ib_uverbs_create_ah(struct ib_uverbs_file *file,
 	struct rdma_ah_attr		attr;
 	int ret;
 	struct ib_udata                   udata;
-	u8				*dmac;
 
 	if (out_len < sizeof resp)
 		return -ENOSPC;

@@ -2564,20 +2563,13 @@ ssize_t ib_uverbs_create_ah(struct ib_uverbs_file *file,
 	} else {
 		rdma_ah_set_ah_flags(&attr, 0);
 	}
-	dmac = rdma_ah_retrieve_dmac(&attr);
-	if (dmac)
-		memset(dmac, 0, ETH_ALEN);
-
-	ah = pd->device->create_ah(pd, &attr, &udata);
 
+	ah = rdma_create_user_ah(pd, &attr, &udata);
 	if (IS_ERR(ah)) {
 		ret = PTR_ERR(ah);
 		goto err_put;
 	}
 
-	ah->device  = pd->device;
-	ah->pd      = pd;
-	atomic_inc(&pd->usecnt);
 	ah->uobject  = uobj;
 	uobj->user_handle = cmd.user_handle;
 	uobj->object = ah;

index de57d6c11a25..4dcfe47c479d 100644
@@ -302,11 +302,13 @@ EXPORT_SYMBOL(ib_dealloc_pd);
 
 /* Address handles */
 
-struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr)
+static struct ib_ah *_rdma_create_ah(struct ib_pd *pd,
+				     struct rdma_ah_attr *ah_attr,
+				     struct ib_udata *udata)
 {
 	struct ib_ah *ah;
 
-	ah = pd->device->create_ah(pd, ah_attr, NULL);
+	ah = pd->device->create_ah(pd, ah_attr, udata);
 
 	if (!IS_ERR(ah)) {
 		ah->device  = pd->device;

@@ -318,8 +320,42 @@ struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr)
 
 	return ah;
 }
+
+struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr)
+{
+	return _rdma_create_ah(pd, ah_attr, NULL);
+}
 EXPORT_SYMBOL(rdma_create_ah);
 
+/**
+ * rdma_create_user_ah - Creates an address handle for the
+ * given address vector.
+ * It resolves destination mac address for ah attribute of RoCE type.
+ * @pd: The protection domain associated with the address handle.
+ * @ah_attr: The attributes of the address vector.
+ * @udata: pointer to user's input output buffer information need by
+ *         provider driver.
+ *
+ * It returns 0 on success and returns appropriate error code on error.
+ * The address handle is used to reference a local or global destination
+ * in all UD QP post sends.
+ */
+struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
+				  struct rdma_ah_attr *ah_attr,
+				  struct ib_udata *udata)
+{
+	int err;
+
+	if (ah_attr->type == RDMA_AH_ATTR_TYPE_ROCE) {
+		err = ib_resolve_eth_dmac(pd->device, ah_attr);
+		if (err)
+			return ERR_PTR(err);
+	}
+
+	return _rdma_create_ah(pd, ah_attr, udata);
+}
+EXPORT_SYMBOL(rdma_create_user_ah);
+
 int ib_get_rdma_header_version(const union rdma_network_hdr *hdr)
 {
 	const struct iphdr *ip4h = (struct iphdr *)&hdr->roce4grh;

index e8608b2dc844..09c4a695155e 100644
@@ -2857,6 +2857,21 @@ void ib_dealloc_pd(struct ib_pd *pd);
  */
 struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr);
 
+/**
+ * rdma_create_user_ah - Creates an address handle for the given address vector.
+ * It resolves destination mac address for ah attribute of RoCE type.
+ * @pd: The protection domain associated with the address handle.
+ * @ah_attr: The attributes of the address vector.
+ * @udata: pointer to user's input output buffer information need by
+ *         provider driver.
+ *
+ * It returns 0 on success and returns appropriate error code on error.
+ * The address handle is used to reference a local or global destination
+ * in all UD QP post sends.
+ */
+struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
+				  struct rdma_ah_attr *ah_attr,
+				  struct ib_udata *udata);
 /**
  * ib_get_gids_from_rdma_hdr - Get sgid and dgid from GRH or IPv4 header
  *   work completion.

2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

From: Parav Pandit <parav-VPRAkNaXOzVWk0Htik3J/[email protected]>

Since IB/core resolves the destination mac address for user and kernel consumers, avoid resolving in multiple provider drivers.

Only ib_core resolves DMAC now, therefore resolve_eth_dmac is removed as exported symbol.

---
 drivers/infiniband/core/verbs.c          |  8 +++++---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c |  8 --------
 drivers/infiniband/hw/hns/hns_roce_ah.c  | 14 +-------------
 drivers/infiniband/hw/mlx4/ah.c          |  8 +++-----
 drivers/infiniband/hw/mlx5/ah.c          |  4 ----
 drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 15 ---------------
 include/rdma/ib_verbs.h                  |  2 --
 7 files changed, 9 insertions(+), 50 deletions(-)

index 4dcfe47c479d..d8f1a5d34f4f 100644
@@ -53,6 +53,9 @@
 
 #include "core_priv.h"
 
+static int ib_resolve_eth_dmac(struct ib_device *device,
+			       struct rdma_ah_attr *ah_attr);
+
 static const char * const ib_events[] = {
 	[IB_EVENT_CQ_ERR]		= "CQ error",
 	[IB_EVENT_QP_FATAL]		= "QP fatal error",

@@ -1257,8 +1260,8 @@ int ib_modify_qp_is_ok(enum ib_qp_state cur_state, enum ib_qp_state next_state,
 }
 EXPORT_SYMBOL(ib_modify_qp_is_ok);
 
-int ib_resolve_eth_dmac(struct ib_device *device,
-			struct rdma_ah_attr *ah_attr)
+static int ib_resolve_eth_dmac(struct ib_device *device,
+			       struct rdma_ah_attr *ah_attr)
 {
 	int           ret = 0;
 	struct ib_global_route *grh;

@@ -1317,7 +1320,6 @@ int ib_resolve_eth_dmac(struct ib_device *device,
 out:
 	return ret;
 }
-EXPORT_SYMBOL(ib_resolve_eth_dmac);
 
 /**
  * ib_modify_qp_with_udata - Modifies the attributes for the specified QP.

index af65f8114379..ebcdfb4f5f75 100644
@@ -729,14 +729,6 @@ struct ib_ah *bnxt_re_create_ah(struct ib_pd *ib_pd,
 			ah->qplib_ah.nw_type = CMDQ_CREATE_AH_TYPE_V1;
 			break;
 		}
-		rc = rdma_addr_find_l2_eth_by_grh(&sgid, &grh->dgid,
-						  ah_attr->roce.dmac, &vlan_tag,
-						  &sgid_attr.ndev->ifindex,
-						  NULL);
-		if (rc) {
-			dev_err(rdev_to_dev(rdev), "Failed to get dmac\n");
-			goto fail;
-		}
 	}
 
 	memcpy(ah->qplib_ah.dmac, ah_attr->roce.dmac, ETH_ALEN);

index b30bc8830643..7dd6a66ea244 100644
@@ -48,7 +48,6 @@ struct ib_ah *hns_roce_create_ah(struct ib_pd *ibpd,
 	struct ib_gid_attr gid_attr;
 	struct hns_roce_ah *ah;
 	u16 vlan_tag = 0xffff;
-	struct in6_addr in6;
 	const struct ib_global_route *grh = rdma_ah_read_grh(ah_attr);
 	union ib_gid sgid;
 	int ret;

@@ -58,18 +57,7 @@ struct ib_ah *hns_roce_create_ah(struct ib_pd *ibpd,
 		return ERR_PTR(-ENOMEM);
 
 	/* Get mac address */
-	memcpy(&in6, grh->dgid.raw, sizeof(grh->dgid.raw));
-	if (rdma_is_multicast_addr(&in6)) {
-		rdma_get_mcast_mac(&in6, ah->av.mac);
-	} else {
-		u8 *dmac = rdma_ah_retrieve_dmac(ah_attr);
-
-		if (!dmac) {
-			kfree(ah);
-			return ERR_PTR(-EINVAL);
-		}
-		memcpy(ah->av.mac, dmac, ETH_ALEN);
-	}
+	memcpy(ah->av.mac, ah_attr->roce.dmac, ETH_ALEN);
 
 	/* Get source gid */
 	ret = ib_get_cached_gid(ibpd->device, rdma_ah_get_port_num(ah_attr),

index 538c46a73248..6dee4fdc5d67 100644
@@ -92,12 +92,10 @@ static struct ib_ah *create_iboe_ah(struct ib_pd *pd,
 	int ret;
 
 	memcpy(&in6, grh->dgid.raw, sizeof(in6));
-	if (rdma_is_multicast_addr(&in6)) {
+	if (rdma_is_multicast_addr(&in6))
 		is_mcast = 1;
-		rdma_get_mcast_mac(&in6, ah->av.eth.mac);
-	} else {
-		memcpy(ah->av.eth.mac, ah_attr->roce.dmac, ETH_ALEN);
-	}
+
+	memcpy(ah->av.eth.mac, ah_attr->roce.dmac, ETH_ALEN);
 	ret = ib_get_cached_gid(pd->device, rdma_ah_get_port_num(ah_attr),
 				grh->sgid_index, &sgid, &gid_attr);
 	if (ret)

index 3363e29157f6..fe269f680103 100644
@@ -89,10 +89,6 @@ struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr,
 
 		resp.response_length = min_resp_len;
 
-		err = ib_resolve_eth_dmac(pd->device, ah_attr);
-		if (err)
-			return ERR_PTR(err);
-
 		memcpy(resp.dmac, ah_attr->roce.dmac, ETH_ALEN);
 		err = ib_copy_to_udata(udata, &resp, resp.response_length);
 		if (err)

index d0249e463338..dec650930ca6 100644
@@ -201,21 +201,6 @@ struct ib_ah *ocrdma_create_ah(struct ib_pd *ibpd, struct rdma_ah_attr *attr,
 	/* Get network header type for this GID */
 	ah->hdr_type = ib_gid_to_network_type(sgid_attr.gid_type, &sgid);
 
-	if ((pd->uctx) &&
-	    (!rdma_is_multicast_addr((struct in6_addr *)grh->dgid.raw)) &&
-	    (!rdma_link_local_addr((struct in6_addr *)grh->dgid.raw))) {
-		status = rdma_addr_find_l2_eth_by_grh(&sgid, &grh->dgid,
-						      attr->roce.dmac,
-						      &vlan_tag,
-						      &sgid_attr.ndev->ifindex,
-						      NULL);
-		if (status) {
-			pr_err("%s(): Failed to resolve dmac from gid." 
-				"status = %d\n", __func__, status);
-			goto av_conf_err;
-		}
-	}
-
 	status = set_av_attr(dev, ah, attr, &sgid, pd->id, &isvlan, vlan_tag);
 	if (status)
 		goto av_conf_err;

index 09c4a695155e..9810e4568635 100644
@@ -3622,8 +3622,6 @@ void ib_drain_rq(struct ib_qp *qp);
 void ib_drain_sq(struct ib_qp *qp);
 void ib_drain_qp(struct ib_qp *qp);
 
-int ib_resolve_eth_dmac(struct ib_device *device,
-			struct rdma_ah_attr *ah_attr);
 int ib_get_eth_speed(struct ib_device *dev, u8 port_num, u8 *speed, u8 *width);
 
 static inline u8 *rdma_ah_retrieve_dmac(struct rdma_ah_attr *attr)

2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

From: Parav Pandit <parav-VPRAkNaXOzVWk0Htik3J/[email protected]>

This patch fixes the case where 'lifespan' entry of the hw_counters is not writable. Currently write callback is not exposed for for the hw_counters sysfs operation. Due to this, modifying lifespan value results into permission denied error in below example.

echo 10 > /sys/class/infiniband/mlx5_0/ports/1/hw_counters/lifespan -bash: /sys/class/infiniband/mlx5_0/ports/1/hw_counters/lifespan:
Permission denied

This patch adds the hook to modify any attribute which implements store() operation.

---
 drivers/infiniband/core/sysfs.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

index abc5ab581f82..e30d86fa1855 100644
@@ -108,8 +108,22 @@ static ssize_t port_attr_show(struct kobject *kobj,
 	return port_attr->show(p, port_attr, buf);
 }
 
+static ssize_t port_attr_store(struct kobject *kobj,
+			       struct attribute *attr,
+			       const char *buf, size_t count)
+{
+	struct port_attribute *port_attr =
+		container_of(attr, struct port_attribute, attr);
+	struct ib_port *p = container_of(kobj, struct ib_port, kobj);
+
+	if (!port_attr->store)
+		return -EIO;
+	return port_attr->store(p, port_attr, buf, count);
+}
+
 static const struct sysfs_ops port_sysfs_ops = {
-	.show = port_attr_show
+	.show	= port_attr_show,
+	.store	= port_attr_store
 };
 
 static ssize_t gid_attr_show(struct kobject *kobj,

2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

From: Parav Pandit <parav-VPRAkNaXOzVWk0Htik3J/[email protected]>

The IB/core provides address resolution service and invokes callback handler when address resolve request completes of requester in worker thread context.

Such caller might allocate or free memory in callback handler depending on the completion status to make further progress or to terminate a connection. Most ULPs resolve route which involves allocating route entry and path record elements in callback event handler.

It has been noticed that WQ_MEM_RECLAIM flag should not be used for workers that tend to allocate memory in this [1] thread discussion.

In order to mitigate this situation, WQ_MEM_RECLAIM flag was dropped for other such WQs in this [2] patch.

Similar problem might arise with address resolution path, though its not yet noticed. The ib_addr workqueue is not memory reclaim path due to its nature of invoking callback that might allocate memory or don't free any memory under memory pressure.

[1] https://www.spinics.net/lists/linux-rdma/msg53239.html
[2] https://www.spinics.net/lists/linux-rdma/msg53416.html

---
 drivers/infiniband/core/addr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 12523f630b61..d2f74721b3ba 100644
@@ -852,7 +852,7 @@ static struct notifier_block nb = {
 
 int addr_init(void)
 {
-	addr_wq = alloc_ordered_workqueue("ib_addr", WQ_MEM_RECLAIM);
+	addr_wq = alloc_ordered_workqueue("ib_addr", 0);
 	if (!addr_wq)
 		return -ENOMEM;
 

2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

This looks like a -stable patch candidate. Once accepted this should probably be queued for v4.9+.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

I don't have any strong opinion about that.

Thanks

AFAICT, this was broken forever because single threaded workqueues are set with MEM_RECLAIM. If anything the Fixes tag is incorrect. --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

You are right. I suggested this Fixes to Parav, because it is needed for stable tree and/or backporting. It makes no sense for me to put Fixes which will point to the code which was completely different before Fixes.

Thanks

From: Parav Pandit <parav-VPRAkNaXOzVWk0Htik3J/[email protected]>

This fix considers optional UDP, XRC header sizes and mandatory ICRC 4 bytes in calculation of path MTU, by considering additional 8 bytes, path MTU calculation is more accurate.

---
 include/rdma/ib_addr.h |  7 ++++---
 include/rdma/ib_pack.h | 19 +++++++++++--------
 2 files changed, 15 insertions(+), 11 deletions(-)

index cfa82d16573d..b2a10c762304 100644
@@ -245,10 +245,11 @@ static inline void rdma_addr_set_dgid(struct rdma_dev_addr *dev_addr, union ib_g
 static inline enum ib_mtu iboe_get_mtu(int mtu)
 {
 	/*
-	 * reduce IB headers from effective IBoE MTU. 28 stands for
-	 * atomic header which is the biggest possible header after BTH
+	 * Reduce IB headers from effective IBoE MTU.
 	 */
-	mtu = mtu - IB_GRH_BYTES - IB_BTH_BYTES - 28;
+	mtu = mtu - (IB_GRH_BYTES + IB_UDP_BYTES + IB_BTH_BYTES +
+		     IB_EXT_XRC_BYTES + IB_EXT_ATOMICETH_BYTES +
+		     IB_ICRC_BYTES);
 
 	if (mtu >= ib_mtu_enum_to_int(IB_MTU_4096))
 		return IB_MTU_4096;

index 36655899ee02..7ea1382ad0e5 100644
@@ -37,14 +37,17 @@
 #include <uapi/linux/if_ether.h>
 
 enum {
-	IB_LRH_BYTES  = 8,
-	IB_ETH_BYTES  = 14,
-	IB_VLAN_BYTES = 4,
-	IB_GRH_BYTES  = 40,
-	IB_IP4_BYTES  = 20,
-	IB_UDP_BYTES  = 8,
-	IB_BTH_BYTES  = 12,
-	IB_DETH_BYTES = 8
+	IB_LRH_BYTES		= 8,
+	IB_ETH_BYTES		= 14,
+	IB_VLAN_BYTES		= 4,
+	IB_GRH_BYTES		= 40,
+	IB_IP4_BYTES		= 20,
+	IB_UDP_BYTES		= 8,
+	IB_BTH_BYTES		= 12,
+	IB_DETH_BYTES		= 8,
+	IB_EXT_ATOMICETH_BYTES	= 28,
+	IB_EXT_XRC_BYTES	= 4,
+	IB_ICRC_BYTES		= 4
 };
 
 struct ib_field {

2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Doug,

The title should be "... optional UDP .." and not as I wrote.

Resubmit one patch/whole series/do nothing?

Thanks.

I'll fix it up.

Doug Ledford <[email protected]>
    GPG KeyID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

From: Jérémy Lefaure <[email protected]>

Using the ARRAY_SIZE macro improves the readability of the code.

Found with Coccinelle with the following semantic patch:
@r depends on (org || report)@
type T;
T[] E;
position p;
@@
(
(sizeof(E)@p /sizeof(*E))
|
(sizeof(E)@p /sizeof(E[...]))
|
(sizeof(E)@p /sizeof(T))
)

---
 drivers/infiniband/hw/mlx5/odp.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

index 3d701c7a4c91..e2197bdda89c 100644
@@ -32,6 +32,7 @@
 
 #include <rdma/ib_umem.h>
 #include <rdma/ib_umem_odp.h>
+#include <linux/kernel.h>
 
 #include "mlx5_ib.h"
 #include "cmd.h"

@@ -929,9 +930,8 @@ static int mlx5_ib_mr_initiator_pfault_handler(
 		return -EFAULT;
 	}
 
-	if (unlikely(opcode >= sizeof(mlx5_ib_odp_opcode_cap) /
-	    sizeof(mlx5_ib_odp_opcode_cap[0]) ||
-	    !(transport_caps & mlx5_ib_odp_opcode_cap[opcode]))) {
+	if (unlikely(opcode >= ARRAY_SIZE(mlx5_ib_odp_opcode_cap) ||
+		     !(transport_caps & mlx5_ib_odp_opcode_cap[opcode]))) {
 		mlx5_ib_err(dev, "ODP fault on QP of an unsupported opcode 0x%x\n",
 			    opcode);
 		return -EFAULT;

2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

This branch is unpullable for me. Trying to pull this on top of the shared pull request brings in a ton of other stuff. My diffstat ends up looking like this:

[dledford@linux-ws infiniband (mlnx-shared)]$ git pull git://git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git tags/rdma-next-2017-10-16
remote: Counting objects: 94, done.
remote: Compressing objects: 100% (72/72), done.
remote: Total 94 (delta 32), reused 30 (delta 21)
Unpacking objects: 100% (94/94), done.

* tag rdma-next-2017-10-16 -> FETCH_HEAD
Auto-merging drivers/infiniband/hw/qedr/qedr_roce_cm.c Merge made by the 'recursive' strategy.

 MAINTAINERS                                        |    1 +
 drivers/infiniband/Kconfig                         |    2 +-
 drivers/infiniband/core/addr.c                     |    2 +-
 drivers/infiniband/core/cm.c                       |   27 +-
 drivers/infiniband/core/cma.c                      |   11 +-
 drivers/infiniband/core/iwcm.c                     |    3 -
 drivers/infiniband/core/rw.c                       |   24 +-
 drivers/infiniband/core/sysfs.c                    |   16 +-
 drivers/infiniband/core/user_mad.c                 |   13 +-
 drivers/infiniband/core/uverbs.h                   |   35 +-
 drivers/infiniband/core/uverbs_cmd.c               |  135 +-
 drivers/infiniband/core/uverbs_ioctl_merge.c       |    2 +-
 drivers/infiniband/core/uverbs_main.c              |   22 +-
 drivers/infiniband/core/uverbs_marshall.c          |   10 +-
 drivers/infiniband/core/uverbs_std_types.c         |    3 +-
 drivers/infiniband/core/verbs.c                    |   48 +-
 drivers/infiniband/hw/bnxt_re/ib_verbs.c           |   20 +-
 drivers/infiniband/hw/bnxt_re/qplib_fp.c           |    7 +-
 drivers/infiniband/hw/bnxt_re/qplib_rcfw.c         |    4 -
 drivers/infiniband/hw/bnxt_re/qplib_sp.c           |    5 +-
 drivers/infiniband/hw/cxgb3/Kconfig                |    2 +-
 drivers/infiniband/hw/cxgb3/cxio_hal.c             |    6 +-
 drivers/infiniband/hw/cxgb3/iwch_cm.c              |    6 +-
 drivers/infiniband/hw/cxgb3/iwch_qp.c              |    3 +
 drivers/infiniband/hw/cxgb4/Kconfig                |    2 +-
 drivers/infiniband/hw/cxgb4/cm.c                   |  292 +-
 drivers/infiniband/hw/cxgb4/cq.c                   |  117 +-
 drivers/infiniband/hw/cxgb4/device.c               |   41 +-
 drivers/infiniband/hw/cxgb4/ev.c                   |    2 +-
 drivers/infiniband/hw/cxgb4/iw_cxgb4.h             |   83 +-
 drivers/infiniband/hw/cxgb4/mem.c                  |  268 +-
 drivers/infiniband/hw/cxgb4/provider.c             |   36 +-
 drivers/infiniband/hw/cxgb4/qp.c                   |  141 +-
 drivers/infiniband/hw/cxgb4/resource.c             |   46 +-
 drivers/infiniband/hw/cxgb4/t4.h                   |   20 +-
 drivers/infiniband/hw/hfi1/chip.c                  |  182 +-
 drivers/infiniband/hw/hfi1/chip.h                  |    1 +
 drivers/infiniband/hw/hfi1/debugfs.c               |    4 +-
 drivers/infiniband/hw/hfi1/driver.c                |   10 +-
 drivers/infiniband/hw/hfi1/file_ops.c              |  486 +--
 drivers/infiniband/hw/hfi1/firmware.c              |   87 +-
 drivers/infiniband/hw/hfi1/hfi.h                   |   26 +-
 drivers/infiniband/hw/hfi1/init.c                  |   43 +-
 drivers/infiniband/hw/hfi1/mad.c                   |   10 +-
 drivers/infiniband/hw/hfi1/rc.c                    |    2 +-
 drivers/infiniband/hw/hfi1/ruc.c                   |    9 -
 drivers/infiniband/hw/hfi1/sdma.c                  |   19 +-
 drivers/infiniband/hw/hfi1/sysfs.c                 |    2 +-
 drivers/infiniband/hw/hfi1/trace.c                 |   23 +-
 drivers/infiniband/hw/hfi1/trace.h                 |   10 +
 drivers/infiniband/hw/hfi1/trace_ibhdrs.h          |   49 +-
 drivers/infiniband/hw/hfi1/trace_rx.h              |   11 +-
 drivers/infiniband/hw/hfi1/ud.c                    |    2 -
 drivers/infiniband/hw/hfi1/user_exp_rcv.c          |    9 +-
 drivers/infiniband/hw/hfi1/user_sdma.c             |   62 +-
 drivers/infiniband/hw/hfi1/user_sdma.h             |   29 +-
 drivers/infiniband/hw/hfi1/verbs.c                 |   10 -
 drivers/infiniband/hw/hfi1/vnic_main.c             |    7 +-
 drivers/infiniband/hw/hns/Kconfig                  |   25 +-
 drivers/infiniband/hw/hns/Makefile                 |    8 +-
 drivers/infiniband/hw/hns/hns_roce_ah.c            |   16 +-
 drivers/infiniband/hw/hns/hns_roce_alloc.c         |    8 +-
 drivers/infiniband/hw/hns/hns_roce_cmd.c           |  107 +-
 drivers/infiniband/hw/hns/hns_roce_cmd.h           |   50 +
 drivers/infiniband/hw/hns/hns_roce_common.h        |   23 +
 drivers/infiniband/hw/hns/hns_roce_cq.c            |   72 +-
 drivers/infiniband/hw/hns/hns_roce_device.h        |  101 +-
 drivers/infiniband/hw/hns/hns_roce_eq.c            |    6 +-
 drivers/infiniband/hw/hns/hns_roce_hem.c           |  699 ++++-
 drivers/infiniband/hw/hns/hns_roce_hem.h           |   32 +-
 drivers/infiniband/hw/hns/hns_roce_hw_v1.c         |  599 +++-
 drivers/infiniband/hw/hns/hns_roce_hw_v1.h         |    5 +
 drivers/infiniband/hw/hns/hns_roce_hw_v2.c         | 3133 ++++++++++++++++++++
 drivers/infiniband/hw/hns/hns_roce_hw_v2.h         | 1165 ++++++++
 drivers/infiniband/hw/hns/hns_roce_main.c          |  342 +--
 drivers/infiniband/hw/hns/hns_roce_mr.c            |  512 +++-
 drivers/infiniband/hw/hns/hns_roce_pd.c            |   20 +-
 drivers/infiniband/hw/hns/hns_roce_qp.c            |  179 +-
 drivers/infiniband/hw/i40iw/Kconfig                |    1 +
 drivers/infiniband/hw/i40iw/i40iw_cm.c             |   20 +-
 drivers/infiniband/hw/i40iw/i40iw_cm.h             |    1 +
 drivers/infiniband/hw/i40iw/i40iw_hw.c             |    1 +
 drivers/infiniband/hw/i40iw/i40iw_puda.c           |    1 +
 drivers/infiniband/hw/i40iw/i40iw_uk.c             |   13 +
 drivers/infiniband/hw/i40iw/i40iw_user.h           |    1 +
 drivers/infiniband/hw/i40iw/i40iw_utils.c          |   13 +-
 drivers/infiniband/hw/i40iw/i40iw_verbs.c          |   13 +
 drivers/infiniband/hw/mlx4/ah.c                    |    8 +-
 drivers/infiniband/hw/mlx4/cq.c                    |    2 +
 drivers/infiniband/hw/mlx4/mcg.c                   |    1 +
 drivers/infiniband/hw/mlx5/ah.c                    |    4 -
 drivers/infiniband/hw/mlx5/cq.c                    |    2 +
 drivers/infiniband/hw/mlx5/mr.c                    |    4 +-
 drivers/infiniband/hw/mlx5/odp.c                   |    6 +-
 drivers/infiniband/hw/mlx5/qp.c                    |    3 +-
 drivers/infiniband/hw/mthca/mthca_main.c           |   10 +-
 drivers/infiniband/hw/nes/nes.c                    |   33 +-
 drivers/infiniband/hw/nes/nes_cm.c                 |    9 +-
 drivers/infiniband/hw/nes/nes_hw.c                 |    7 +-
 drivers/infiniband/hw/nes/nes_nic.c                |    9 +-
 drivers/infiniband/hw/nes/nes_utils.c              |   13 +-
 drivers/infiniband/hw/nes/nes_verbs.c              |   12 +-
 drivers/infiniband/hw/ocrdma/ocrdma_ah.c           |   15 -
 drivers/infiniband/hw/ocrdma/ocrdma_hw.c           |    8 +-
 drivers/infiniband/hw/ocrdma/ocrdma_stats.c        |    2 +-
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c        |    4 +-
 drivers/infiniband/hw/qedr/Kconfig                 |    1 +
 drivers/infiniband/hw/qedr/Makefile                |    2 +-
 drivers/infiniband/hw/qedr/main.c                  |  118 +-
 drivers/infiniband/hw/qedr/qedr.h                  |   31 +-
 drivers/infiniband/hw/qedr/qedr_hsi_rdma.h         |    6 +-
 drivers/infiniband/hw/qedr/qedr_iw_cm.c            |  749 +++++
 drivers/infiniband/hw/qedr/qedr_iw_cm.h            |   49 +
 .../hw/qedr/{qedr_cm.c => qedr_roce_cm.c}          |   31 +-
 .../hw/qedr/{qedr_cm.h => qedr_roce_cm.h}          |    0
 drivers/infiniband/hw/qedr/verbs.c                 |  359 ++-
 drivers/infiniband/hw/qedr/verbs.h                 |    2 +
 drivers/infiniband/hw/qib/Kconfig                  |    1 +
 drivers/infiniband/hw/qib/qib.h                    |   26 +-
 drivers/infiniband/hw/qib/qib_7220.h               |    1 -
 drivers/infiniband/hw/qib/qib_diag.c               |    6 -
 drivers/infiniband/hw/qib/qib_driver.c             |    8 +-
 drivers/infiniband/hw/qib/qib_file_ops.c           |    9 -
 drivers/infiniband/hw/qib/qib_iba6120.c            |   62 +-
 drivers/infiniband/hw/qib/qib_iba7220.c            |   75 +-
 drivers/infiniband/hw/qib/qib_iba7322.c            |  179 +-
 drivers/infiniband/hw/qib/qib_mad.c                |   14 +-
 drivers/infiniband/hw/qib/qib_pcie.c               |  128 +-
 drivers/infiniband/hw/qib/qib_rc.c                 |    2 +-
 drivers/infiniband/hw/qib/qib_sd7220.c             |    2 -
 drivers/infiniband/hw/qib/qib_sdma.c               |    2 +-
 drivers/infiniband/hw/qib/qib_tx.c                 |    4 -
 drivers/infiniband/hw/qib/qib_verbs.c              |    3 +-
 drivers/infiniband/hw/usnic/usnic_ib_qp_grp.c      |    2 -
 drivers/infiniband/hw/usnic/usnic_ib_qp_grp.h      |   25 +-
 drivers/infiniband/hw/usnic/usnic_ib_sysfs.c       |    1 +
 drivers/infiniband/hw/usnic/usnic_ib_verbs.c       |   25 +
 drivers/infiniband/sw/rdmavt/Kconfig               |    1 +
 drivers/infiniband/sw/rdmavt/qp.c                  |    1 +
 drivers/infiniband/sw/rxe/rxe_comp.c               |    4 +-
 drivers/infiniband/sw/rxe/rxe_pool.c               |   16 +-
 drivers/infiniband/sw/rxe/rxe_task.c               |    2 +-
 drivers/infiniband/sw/rxe/rxe_verbs.c              |    1 +
 drivers/infiniband/ulp/ipoib/ipoib_cm.c            |   10 +-
 drivers/infiniband/ulp/ipoib/ipoib_ib.c            |    8 +-
 drivers/infiniband/ulp/ipoib/ipoib_main.c          |    4 +-
 drivers/infiniband/ulp/iser/iser_verbs.c           |    2 +-
 drivers/infiniband/ulp/isert/ib_isert.c            |   14 +-
 drivers/infiniband/ulp/opa_vnic/opa_vnic_encap.c   |   42 +-
 drivers/infiniband/ulp/opa_vnic/opa_vnic_encap.h   |   22 +-
 .../infiniband/ulp/opa_vnic/opa_vnic_internal.h    |    7 +-
 drivers/infiniband/ulp/opa_vnic/opa_vnic_netdev.c  |   44 +-
 drivers/infiniband/ulp/opa_vnic/opa_vnic_vema.c    |    1 +
 .../infiniband/ulp/opa_vnic/opa_vnic_vema_iface.c  |   22 +-
 drivers/infiniband/ulp/srp/ib_srp.c                |    2 +-
 drivers/net/ethernet/chelsio/cxgb3/t3cdev.h        |    2 +-
 drivers/net/ethernet/mellanox/mlx5/core/fs_core.c  |  853 ++++--
 drivers/net/ethernet/mellanox/mlx5/core/fs_core.h  |   11 +-
 drivers/staging/lustre/lnet/Kconfig                |    2 +-
 include/rdma/ib_addr.h                             |   11 +-
 include/rdma/ib_pack.h                             |   19 +-
 include/rdma/ib_sa.h                               |   12 +-
 include/rdma/ib_verbs.h                            |   17 +-
 include/rdma/opa_addr.h                            |    6 +-
 include/rdma/rdmavt_qp.h                           |    6 +-
 165 files changed, 10122 insertions(+), 2809 deletions(-)
 create mode 100644 drivers/infiniband/hw/hns/hns_roce_hw_v2.c
 create mode 100644 drivers/infiniband/hw/hns/hns_roce_hw_v2.h
 create mode 100644 drivers/infiniband/hw/qedr/qedr_iw_cm.c
 create mode 100644 drivers/infiniband/hw/qedr/qedr_iw_cm.h
 rename drivers/infiniband/hw/qedr/{qedr_cm.c => qedr_roce_cm.c} (96%)
 rename drivers/infiniband/hw/qedr/{qedr_cm.h => qedr_roce_cm.h} (100%)

So, with a shared pull request, your submissions either need to be against that shared pull request, or against my for-next, and you need to let me know which one it is.

I can grab these out of patchworks, but even that way I still need you to be clear about what tree/branch these are intended to be applied to.

Doug Ledford <[email protected]>
    GPG KeyID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

<....>

I didn't know that you are using this as a pull request, we tried it in last cycle and it didn't truly work, so I stopped it and changed the title (instead of pull requests, it is patches now), removed invitation to pull from the cover letter and prepared the series (with tags) as standalone patches and not as pull request.

It should apply cleanly as patches grabbed from the patchworks and if for any reasons it doesn't work for you, please drop it and I'll resubmit.

Pull requests model works for truly serial submission and it is not the case now.

Thanks

OK, but you still had the pull request URL in the cover letter, so I tried it. Anyway, not a big deal as far as that goes.

Apply cleanly where? That was the other part of my email. When you have a shared pull request, and it isn't based on my for-next branch (as this one isn't), please let me know in each patch series whether you intend it to be applied on the shared pull request or my for-next branch.

I actually applied it to the shared pull request branch, but I didn't know if that's what you had intended. Also, I rewrote the commit message in patch 5/6 entirely. But, otherwise, thanks, applied.

Doug Ledford <[email protected]>
    GPG KeyID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBsaW51eC1yZG1hLW93bmVyQHZn ZXIua2VybmVsLm9yZyBbbWFpbHRvOmxpbnV4LXJkbWEtDQo+IG93bmVyQHZnZXIua2VybmVsLm9y Z10gT24gQmVoYWxmIE9mIERvdWcgTGVkZm9yZA0KPiBTZW50OiBXZWRuZXNkYXksIE9jdG9iZXIg MTgsIDIwMTcgMTE6MjAgQU0NCj4gVG86IExlb24gUm9tYW5vdnNreSA8bGVvbkBrZXJuZWwub3Jn Pg0KPiBDYzogbGludXgtcmRtYUB2Z2VyLmtlcm5lbC5vcmcNCj4gU3ViamVjdDogUmU6IFtQQVRD SCByZG1hLW5leHQgdjEgMC82XSBSRE1BIGNvcmUgYW5kIG1seDUgZml4ZXMgYW5kDQo+IHJlZmFj dG9yaW5nIGZvciA0LjE1DQo+IA0KDQo+IEkgYWN0dWFsbHkgYXBwbGllZCBpdCB0byB0aGUgc2hh cmVkIHB1bGwgcmVxdWVzdCBicmFuY2gsIGJ1dCBJIGRpZG4ndCBrbm93IGlmIHRoYXQncw0KPiB3 aGF0IHlvdSBoYWQgaW50ZW5kZWQuICBBbHNvLCBJIHJld3JvdGUgdGhlIGNvbW1pdCBtZXNzYWdl IGluIHBhdGNoIDUvNg0KPiBlbnRpcmVseS4gIEJ1dCwgb3RoZXJ3aXNlLCB0aGFua3MsIGFwcGxp ZWQuDQo+IA0KVGhhbmtzIERvdWcsIEkgcmVhbGl6ZWQgbGF0ZWx5IHRoYXQgOCBieXRlcyBkZXNj cmlwdGlvbiB3YXMgc3RhbGUgd2l0aCBpbmNsdXNpb24gb2YgVURQIGhlYWRlciBvZiB2MS4NCg0K --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

You did the right thing by applying it on shared pull request, thanks. All my submissions are based on shared pull request, just to be on the safe side.

Thanks again, I appreciate your help.