Skip to contentSkip to conversation

Expose CQ moderation to user space

PATCH In review Leon Romanovsky 4 participants 18 messages Oct 19, 2017 – Oct 25, 2017 7 messages not in the archive

Subsystems (5)
InfiniBand Mellanox mlx5 core Mellanox MLX5 IB driver Mellanox MLX4 core VPI driver +1 more subsystems
lore
Share & tools
lore

Leon Romanovsky

author
#0 lore raw

cover Expose CQ moderation to user space

This patch set exposes CQ moderation. This will allow to moderate number of CQEs needed to create an event. Such change brings performance improvement by reducing pressure on application to receive event per-CQE.

The proposed semantics follows the well-established kernel semantics.

* cq_max_count - defines the number of cookies needed to create an event.
* cq_period - defines the timeout between last event and a new one
that will occur even if cq_max_count was not reached

The patches are available in the git repository at:
git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git tags/rdma-next-2017-10-19-2

Thanks
---------------------------------------

Yonatan Cohen (6):
IB/uverbs: Allow CQ moderation with modify CQ IB/mlx4: Exposing modify CQ callback to uverbs layer IB/mlx5: Exposing modify CQ callback to uverbs layer IB/uverbs: Add CQ moderation capability to query_device IB/mlx4: Add CQ moderation capability to query_device IB/mlx5: Add CQ moderation capability to query_device

 drivers/infiniband/core/uverbs.h      |  1 +
 drivers/infiniband/core/uverbs_cmd.c  | 45 +++++++++++++++++++++++++++++++++++
 drivers/infiniband/core/uverbs_main.c |  1 +
 drivers/infiniband/hw/mlx4/main.c     |  6 +++++
 drivers/infiniband/hw/mlx4/mlx4_ib.h  |  1 +
 drivers/infiniband/hw/mlx5/cq.c       |  3 +++
 drivers/infiniband/hw/mlx5/main.c     | 10 +++++++-
 include/linux/mlx4/cq.h               |  3 +++
 include/linux/mlx5/cq.h               |  3 +++
 include/rdma/ib_verbs.h               | 10 ++++++++
 include/uapi/rdma/ib_user_verbs.h     | 18 +++++++++++++-
 11 files changed, 99 insertions(+), 2 deletions(-)
Signature · 6 lines
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20171019053524.11135-1-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org

author
#2 lore raw

PATCH 1/6 IB/uverbs: Allow CQ moderation with modify CQ

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

Uverbs support in modify_cq for CQ moderation only. Gives ability to change cq_max_count and cq_period.
CQ moderation enhance performance by moderating the number of cookies needed to create an event instead of application having to suffer from event per cookie.
To achieve CQ moderation the application needs to set cq_max_count and cq_period.
cq_max_count - defines the number of cookies
needed to create an event.
cq_period - defines the timeout (micro seconds) between last
event and a new one that will occur even if cq_max_count was not satisfied

Signed-off-by Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/core/uverbs.h      |  1 +
 drivers/infiniband/core/uverbs_cmd.c  | 36 +++++++++++++++++++++++++++++++++++
 drivers/infiniband/core/uverbs_main.c |  1 +
 include/rdma/ib_verbs.h               |  4 ++++
 include/uapi/rdma/ib_user_verbs.h     | 11 ++++++++++-
 5 files changed, 52 insertions(+), 1 deletion(-)
drivers/infiniband/core/uverbs.h +1 -0
index ee2739ae4305..deccefb71a6b 100644
@@ -306,5 +306,6 @@ IB_UVERBS_DECLARE_EX_CMD(destroy_wq);
 IB_UVERBS_DECLARE_EX_CMD(create_rwq_ind_table);
 IB_UVERBS_DECLARE_EX_CMD(destroy_rwq_ind_table);
 IB_UVERBS_DECLARE_EX_CMD(modify_qp);
+IB_UVERBS_DECLARE_EX_CMD(modify_cq);
 
 #endif /* UVERBS_H */
drivers/infiniband/core/uverbs_cmd.c +36 -0
index ab29a0327831..c7819e5540f1 100644
@@ -3856,3 +3856,39 @@ int ib_uverbs_ex_query_device(struct ib_uverbs_file *file,
 	err = ib_copy_to_udata(ucore, &resp, resp.response_length);
 	return err;
 }
+
+int ib_uverbs_ex_modify_cq(struct ib_uverbs_file *file,
+			   struct ib_device *ib_dev,
+			   struct ib_udata *ucore,
+			   struct ib_udata *uhw)
+{
+	struct ib_uverbs_ex_modify_cq cmd = {};
+	struct ib_cq *cq;
+	int ret;
+
+	/* sanity checks */
+	if (ucore->inlen > sizeof(cmd) &&
+	    !ib_is_udata_cleared(ucore, sizeof(cmd),
+				 ucore->inlen - sizeof(cmd)))
+		return -EOPNOTSUPP;
+
+	ret = ib_copy_from_udata(&cmd, ucore, min(sizeof(cmd), ucore->inlen));
+	if (ret)
+		return ret;
+
+	if (cmd.attr_mask != IB_CQ_MODERATE)
+		return -EINVAL;
+
+	if (!cmd.comp_mask)
+		return -EINVAL;
+
+	cq = uobj_get_obj_read(cq, cmd.cq_handle, file->ucontext);
+	if (!cq)
+		return -EINVAL;
+
+	ret = ib_modify_cq(cq, cmd.cq_count, cmd.cq_period);
+
+	uobj_put_obj_read(cq);
+
+	return ret;
+}
drivers/infiniband/core/uverbs_main.c +1 -0
index b5febfd84ee5..381fd9c096ae 100644
@@ -128,6 +128,7 @@ static int (*uverbs_ex_cmd_table[])(struct ib_uverbs_file *file,
 	[IB_USER_VERBS_EX_CMD_CREATE_RWQ_IND_TBL] = ib_uverbs_ex_create_rwq_ind_table,
 	[IB_USER_VERBS_EX_CMD_DESTROY_RWQ_IND_TBL] = ib_uverbs_ex_destroy_rwq_ind_table,
 	[IB_USER_VERBS_EX_CMD_MODIFY_QP]        = ib_uverbs_ex_modify_qp,
+	[IB_USER_VERBS_EX_CMD_MODIFY_CQ]        = ib_uverbs_ex_modify_cq,
 };
 
 static void ib_uverbs_add_one(struct ib_device *device);
include/rdma/ib_verbs.h +4 -0
index 4c0a539cd2a2..ec80fec2d113 100644
@@ -311,6 +311,10 @@ struct ib_cq_init_attr {
 	u32		flags;
 };
 
+enum ib_cq_attr_mask {
+	IB_CQ_MODERATE = 1 << 0,
+};
+
 struct ib_device_attr {
 	u64			fw_ver;
 	__be64			sys_image_guid;
include/uapi/rdma/ib_user_verbs.h +10 -1
index d4e0b53bfc75..854aaafc60d0 100644
@@ -100,7 +100,8 @@ enum {
 	IB_USER_VERBS_EX_CMD_MODIFY_WQ,
 	IB_USER_VERBS_EX_CMD_DESTROY_WQ,
 	IB_USER_VERBS_EX_CMD_CREATE_RWQ_IND_TBL,
-	IB_USER_VERBS_EX_CMD_DESTROY_RWQ_IND_TBL
+	IB_USER_VERBS_EX_CMD_DESTROY_RWQ_IND_TBL,
+	IB_USER_VERBS_EX_CMD_MODIFY_CQ
 };
 
 /*

@@ -1150,6 +1151,14 @@ struct ib_uverbs_ex_destroy_rwq_ind_table  {
 	__u32 ind_tbl_handle;
 };
 
+struct ib_uverbs_ex_modify_cq {
+	__u32 cq_handle;
+	__u32 attr_mask;
+	__u16 cq_count;
+	__u16 cq_period;
+	__u32 comp_mask;
+};
+
 #define IB_DEVICE_NAME_MAX 64
 
 #endif /* IB_USER_VERBS_H */
Signature · 6 lines
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20171019053524.11135-2-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org

author
#4 lore raw

Re: [PATCH rdma-next 1/6] IB/uverbs: Allow CQ moderation with modify CQ

Quoted 60 lines · Leon Romanovsky

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

Uverbs support in modify_cq for CQ moderation only. Gives ability to change cq_max_count and cq_period.
CQ moderation enhance performance by moderating the number of cookies needed to create an event instead of application having to suffer from event per cookie.
To achieve CQ moderation the application needs to set cq_max_count and cq_period.
cq_max_count - defines the number of cookies
needed to create an event.
cq_period - defines the timeout (micro seconds) between last
event and a new one that will occur even if cq_max_count was not satisfied

Signed-off-by Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/core/uverbs.h      |  1 +
 drivers/infiniband/core/uverbs_cmd.c  | 36 +++++++++++++++++++++++++++++++++++
 drivers/infiniband/core/uverbs_main.c |  1 +
 include/rdma/ib_verbs.h               |  4 ++++
 include/uapi/rdma/ib_user_verbs.h     | 11 ++++++++++-
 5 files changed, 52 insertions(+), 1 deletion(-)
drivers/infiniband/core/uverbs.h +1 -0
index ee2739ae4305..deccefb71a6b 100644
@@ -306,5 +306,6 @@ IB_UVERBS_DECLARE_EX_CMD(destroy_wq);
 IB_UVERBS_DECLARE_EX_CMD(create_rwq_ind_table);
 IB_UVERBS_DECLARE_EX_CMD(destroy_rwq_ind_table);
 IB_UVERBS_DECLARE_EX_CMD(modify_qp);
+IB_UVERBS_DECLARE_EX_CMD(modify_cq);
 
 #endif /* UVERBS_H */
drivers/infiniband/core/uverbs_cmd.c +15 -0
index ab29a0327831..c7819e5540f1 100644
@@ -3856,3 +3856,39 @@ int ib_uverbs_ex_query_device(struct ib_uverbs_file *file,
 	err = ib_copy_to_udata(ucore, &resp, resp.response_length);
 	return err;
 }
+
+int ib_uverbs_ex_modify_cq(struct ib_uverbs_file *file,
+			   struct ib_device *ib_dev,
+			   struct ib_udata *ucore,
+			   struct ib_udata *uhw)
+{
+	struct ib_uverbs_ex_modify_cq cmd = {};
+	struct ib_cq *cq;
+	int ret;
+
+	/* sanity checks */
+	if (ucore->inlen > sizeof(cmd) &&
+	    !ib_is_udata_cleared(ucore, sizeof(cmd),
+				 ucore->inlen - sizeof(cmd)))
+		return -EOPNOTSUPP;

drivers/infiniband/core/uverbs_cmd.c int ib_uverbs_ex_query_device(struct ib_uverbs_file *file, · line 3883

Quoted 10 lines
drivers/infiniband/core/uverbs_cmd.c +10 -0
+
+	ret = ib_copy_from_udata(&cmd, ucore, min(sizeof(cmd), ucore->inlen));
+	if (ret)
+		return ret;
+
+	if (cmd.attr_mask != IB_CQ_MODERATE)
+		return -EINVAL;
+
+	if (!cmd.comp_mask)
+		return -EINVAL;

No way that it worked, comp_mask is never set, so it is supposed to be zero and it is always returning EINVAL.

Additionally, the "attr_mask" is already carrying the same information as "comp_mask" will handle, hence the "comp_mask" should be dropped.

Thanks

#5 lore raw

Re: [PATCH rdma-next 1/6] IB/uverbs: Allow CQ moderation with modify CQ

In reply to a message not in the archive
Quoted 6 lines · Leon Romanovsky

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

Uverbs support in modify_cq for CQ moderation only. Gives ability to change cq_max_count and cq_period. CQ moderation enhance performance by moderating the number of cookies

^ Please use non-slang in your commit messages. People not
familiar with RDMA will not know what cookies means.

Signature · 8 lines
Doug Ledford <[email protected]>
    GPG KeyID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#7 lore raw

Re: [PATCH rdma-next 1/6] IB/uverbs: Allow CQ moderation with modify CQ

Quoted 9 lines · Doug Ledford
Quoted 6 lines · Leon Romanovsky

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

Uverbs support in modify_cq for CQ moderation only. Gives ability to change cq_max_count and cq_period. CQ moderation enhance performance by moderating the number of cookies

^ Please use non-slang in your commit messages. People not
familiar with RDMA will not know what cookies means.

I'll do my best.

Thanks

Quoted 6 lines

--
Doug Ledford <[email protected]>

    GPG KeyID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD
author
#8 lore raw

PATCH 2/6 IB/mlx4: Exposing modify CQ callback to uverbs layer

In reply to a message not in the archive

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

Exposed mlx4_ib_modify_cq to be called from ib device verb list.

Signed-off-by Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/hw/mlx4/main.c    | 3 +++
 drivers/infiniband/hw/mlx4/mlx4_ib.h | 1 +
 include/linux/mlx4/cq.h              | 3 +++
 3 files changed, 7 insertions(+)
drivers/infiniband/hw/mlx4/main.c +3 -0
index c636842c5be0..58845e67dad1 100644
@@ -2733,6 +2733,9 @@ static void *mlx4_ib_add(struct mlx4_dev *dev)
 	ibdev->ib_dev.get_dev_fw_str    = get_fw_ver_str;
 	ibdev->ib_dev.disassociate_ucontext = mlx4_ib_disassociate_ucontext;
 
+	ibdev->ib_dev.uverbs_ex_cmd_mask |=
+		(1ull << IB_USER_VERBS_EX_CMD_MODIFY_CQ);
+
 	if ((dev->caps.flags2 & MLX4_DEV_CAP_FLAG2_RSS) &&
 	    ((mlx4_ib_port_link_layer(&ibdev->ib_dev, 1) ==
 	    IB_LINK_LAYER_ETHERNET) ||
drivers/infiniband/hw/mlx4/mlx4_ib.h +1 -0
index 1fa19820355a..9a55064ed69a 100644
@@ -47,6 +47,7 @@
 #include <linux/mlx4/device.h>
 #include <linux/mlx4/doorbell.h>
 #include <linux/mlx4/qp.h>
+#include <linux/mlx4/cq.h>
 
 #define MLX4_IB_DRV_NAME	"mlx4_ib"
include/linux/mlx4/cq.h +3 -0
index 09cebe528488..508e8cc5ee86 100644
@@ -136,6 +136,9 @@ enum {
 	MLX4_CQE_BAD_FCS                 = 1 << 4,
 };
 
+#define MLX4_MAX_CQ_PERIOD (BIT(16) - 1)
+#define MLX4_MAX_CQ_COUNT (BIT(16) - 1)
+
 static inline void mlx4_cq_arm(struct mlx4_cq *cq, u32 cmd,
 			       void __iomem *uar_page,
 			       spinlock_t *doorbell_lock)
Signature · 6 lines
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#9 lore raw

PATCH 3/6 IB/mlx5: Exposing modify CQ callback to uverbs layer

In reply to a message not in the archive

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

Exposed mlx5_ib_modify_cq to be called from ib device verb list.

Signed-off-by Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/hw/mlx5/cq.c   | 3 +++
 drivers/infiniband/hw/mlx5/main.c | 3 ++-
 include/linux/mlx5/cq.h           | 3 +++
 3 files changed, 8 insertions(+), 1 deletion(-)
drivers/infiniband/hw/mlx5/cq.c +3 -0
index 01b218a3c277..18705cbcdc8c 100644
@@ -1149,6 +1149,9 @@ int mlx5_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period)
 	if (!MLX5_CAP_GEN(dev->mdev, cq_moderation))
 		return -ENOSYS;
 
+	if (cq_period > MLX5_MAX_CQ_PERIOD)
+		return -EINVAL;
+
 	err = mlx5_core_modify_cq_moderation(dev->mdev, &mcq->mcq,
 					     cq_period, cq_count);
 	if (err)
drivers/infiniband/hw/mlx5/main.c +2 -1
index c44dd0896f80..da4c577d653c 100644
@@ -4020,7 +4020,8 @@ static void *mlx5_ib_add(struct mlx5_core_dev *mdev)
 		(1ull << IB_USER_VERBS_EX_CMD_QUERY_DEVICE)	|
 		(1ull << IB_USER_VERBS_EX_CMD_CREATE_CQ)	|
 		(1ull << IB_USER_VERBS_EX_CMD_CREATE_QP)	|
-		(1ull << IB_USER_VERBS_EX_CMD_MODIFY_QP);
+		(1ull << IB_USER_VERBS_EX_CMD_MODIFY_QP)	|
+		(1ull << IB_USER_VERBS_EX_CMD_MODIFY_CQ);
 
 	dev->ib_dev.query_device	= mlx5_ib_query_device;
 	dev->ib_dev.query_port		= mlx5_ib_query_port;
include/linux/mlx5/cq.h +3 -0
index cc718e245b1e..6be357b219ec 100644
@@ -128,6 +128,9 @@ enum {
 	CQE_SIZE_128_PAD = 2,
 };
 
+#define MLX5_MAX_CQ_PERIOD (BIT(__mlx5_bit_sz(cqc, cq_period)) - 1)
+#define MLX5_MAX_CQ_COUNT (BIT(__mlx5_bit_sz(cqc, cq_max_count)) - 1)
+
 static inline int cqe_sz_to_mlx_sz(u8 size, int padding_128_en)
 {
 	return padding_128_en ? CQE_SIZE_128_PAD :
Signature · 6 lines
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#10 lore raw

PATCH 4/6 IB/uverbs: Add CQ moderation capability to query_device

In reply to a message not in the archive

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

The query_device function can now obtain the maximum values for cq_max_count and cq_period, needed for CQ moderation. cq_max_count is a 16 bits number that determines the number of cookies to accumulate before generating an event. cq_period is a 16 bits number that detemines the timeout in micro seconds from the last event generated, upon which a new event will be generated even if cq_max_count was not reached.

Signed-off-by Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/core/uverbs_cmd.c | 9 +++++++++
 include/rdma/ib_verbs.h              | 6 ++++++
 include/uapi/rdma/ib_user_verbs.h    | 7 +++++++
 3 files changed, 22 insertions(+)
drivers/infiniband/core/uverbs_cmd.c +9 -0
index c7819e5540f1..ce969cc84494 100644
@@ -3852,6 +3852,15 @@ int ib_uverbs_ex_query_device(struct ib_uverbs_file *file,
 	resp.tm_caps.max_sge		= attr.tm_caps.max_sge;
 	resp.tm_caps.flags		= attr.tm_caps.flags;
 	resp.response_length += sizeof(resp.tm_caps);
+
+	if (ucore->outlen < resp.response_length + sizeof(resp.cq_moderation_caps))
+		goto end;
+
+	resp.cq_moderation_caps.max_cq_moderation_count  =
+		attr.cq_caps.max_cq_moderation_count;
+	resp.cq_moderation_caps.max_cq_moderation_period =
+		attr.cq_caps.max_cq_moderation_period;
+	resp.response_length += sizeof(resp.cq_moderation_caps);
 end:
 	err = ib_copy_to_udata(ucore, &resp, resp.response_length);
 	return err;
include/rdma/ib_verbs.h +6 -0
index ec80fec2d113..e1d027acddbd 100644
@@ -315,6 +315,11 @@ enum ib_cq_attr_mask {
 	IB_CQ_MODERATE = 1 << 0,
 };
 
+struct ib_cq_caps {
+	u16     max_cq_moderation_count;
+	u16     max_cq_moderation_period;
+};
+
 struct ib_device_attr {
 	u64			fw_ver;
 	__be64			sys_image_guid;

@@ -365,6 +370,7 @@ struct ib_device_attr {
 	u32			max_wq_type_rq;
 	u32			raw_packet_caps; /* Use ib_raw_packet_caps enum */
 	struct ib_tm_caps	tm_caps;
+	struct ib_cq_caps       cq_caps;
 };
 
 enum ib_mtu {
include/uapi/rdma/ib_user_verbs.h +7 -0
index 854aaafc60d0..7745a0eea57c 100644
@@ -125,6 +125,12 @@ struct ib_uverbs_comp_event_desc {
 	__u64 cq_handle;
 };
 
+struct ib_uverbs_cq_caps {
+	__u16     max_cq_moderation_count;
+	__u16     max_cq_moderation_period;
+	__u32     reserved;
+};
+
 /*
  * All commands from userspace should start with a __u32 command field
  * followed by __u16 in_words and out_words fields (which give the

@@ -263,6 +269,7 @@ struct ib_uverbs_ex_query_device_resp {
 	__u32  max_wq_type_rq;
 	__u32 raw_packet_caps;
 	struct ib_uverbs_tm_caps tm_caps;
+	struct ib_uverbs_cq_caps cq_moderation_caps;
 };
 
 struct ib_uverbs_query_port {
Signature · 6 lines
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#11 lore raw

PATCH 5/6 IB/mlx4: Add CQ moderation capability to query_device

In reply to a message not in the archive

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

query_device can now obtain the maximum values for cq_max_count and cq_period, needed for cq moderation.

Signed-off-by Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/hw/mlx4/main.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/infiniband/hw/mlx4/main.c +3 -0
index 58845e67dad1..c6420e26f2d8 100644
@@ -563,6 +563,9 @@ static int mlx4_ib_query_device(struct ib_device *ibdev,
 		props->max_wq_type_rq = props->max_qp;
 	}
 
+	props->cq_caps.max_cq_moderation_count = MLX4_MAX_CQ_COUNT;
+	props->cq_caps.max_cq_moderation_period = MLX4_MAX_CQ_PERIOD;
+
 	if (!mlx4_is_slave(dev->dev))
 		err = mlx4_get_internal_clock_params(dev->dev, &clock_params);
Signature · 6 lines
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#12 lore raw

PATCH 6/6 IB/mlx5: Add CQ moderation capability to query_device

In reply to a message not in the archive

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>

query_device can now obtain the maximum values for cq_max_count and cq_period, needed for cq moderation.

Signed-off-by Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/[email protected]>
Reviewed-by Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/hw/mlx5/main.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/infiniband/hw/mlx5/main.c +7 -0
index da4c577d653c..61ce3ca503b8 100644
@@ -790,6 +790,13 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 		props->tm_caps.max_sge = MLX5_TM_MAX_SGE;
 	}
 
+	if (MLX5_CAP_GEN(dev->mdev, cq_moderation)) {
+		props->cq_caps.max_cq_moderation_count =
+						MLX5_MAX_CQ_COUNT;
+		props->cq_caps.max_cq_moderation_period =
+						MLX5_MAX_CQ_PERIOD;
+	}
+
 	if (field_avail(typeof(resp), cqe_comp_caps, uhw->outlen)) {
 		resp.cqe_comp_caps.max_num =
 			MLX5_CAP_GEN(dev->mdev, cqe_compression) ?
Signature · 6 lines
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#13 lore raw
In reply to a message not in the archive
Quoted 10 lines · Leon Romanovsky

This patch set exposes CQ moderation. This will allow to moderate number of CQEs needed to create an event. Such change brings performance improvement by reducing pressure on application to receive event per-CQE.

The proposed semantics follows the well-established kernel semantics.

* cq_max_count - defines the number of cookies needed to create an event.
* cq_period - defines the timeout between last event and a new one
that will occur even if cq_max_count was not reached

Quoted 6 lines

The patches are available in the git repository at:
git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git tags/rdma-next-2017-10-19-2

Thanks
---------------------------------------

Doug,

Please drop this series.

I'll resubmit it once user space part will be finished.

Mellanox,
I would like to use this opportunity and to ask publicly, please submit kernel and user spaces altogether.

It causes for too much errors and it is actually wrong to submit it separately.

Thanks

A message that is not in the archive (sent off-list or not yet received): 20171023094744.GH2106-U/[email protected]

#15 lore raw
Quoted 22 lines · Leon Romanovsky
Quoted 10 lines · Leon Romanovsky

This patch set exposes CQ moderation. This will allow to moderate number of CQEs needed to create an event. Such change brings performance improvement by reducing pressure on application to receive event per-CQE.

The proposed semantics follows the well-established kernel semantics.

* cq_max_count - defines the number of cookies needed to create an event.
* cq_period - defines the timeout between last event and a new one
that will occur even if cq_max_count was not reached

Quoted 6 lines

The patches are available in the git repository at:
git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git tags/rdma-next-2017-10-19-2

Thanks
---------------------------------------

Doug,

Please drop this series.

Quoted 8 lines

I'll resubmit it once user space part will be finished.

Mellanox,
I would like to use this opportunity and to ask publicly, please submit kernel and user spaces altogether.

It causes for too much errors and it is actually wrong to submit it separately.

Hi Leon,

I have a generic question. And I apologize if this is redundant and has been answered before. But what is the process for user-space submissions?

For libi40iw we make all submissions to this mailing list _only_ and this is all we are aware of. Are there alternate submission paths?

Shiraz

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20171023130804.GA21584-GOXS9JX10wfOxmVO0tvppfooFf0ArEBIu+b9c/[email protected]

#17 lore raw
Quoted 2 lines · Shiraz Saleem

For libi40iw we make all submissions to this mailing list _only_ and this is all we are aware of. Are there alternate submission paths?

It works best if you send a github pull request to

https://github.com/linux-rdma/rdma-core

At the same time you make your mailing list submission, then we can see that the patches pass CI and it is less work for the maintainers.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20171023152919.GC11952-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org

#19 lore raw
Quoted 5 lines · Jason Gunthorpe
Quoted 2 lines · Shiraz Saleem

For libi40iw we make all submissions to this mailing list _only_ and this is all we are aware of. Are there alternate submission paths?

Quoted 8 lines

It works best if you send a github pull request to

https://github.com/linux-rdma/rdma-core

At the same time you make your mailing list submission, then we can see that the patches pass CI and it is less work for the maintainers.

Jason

OK. Thanks!

Shiraz
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#20 lore raw
In reply to a message not in the archive
Quoted 29 lines · Shiraz Saleem
Quoted 28 lines · Leon Romanovsky
Quoted 10 lines · Leon Romanovsky

This patch set exposes CQ moderation. This will allow to moderate number of CQEs needed to create an event. Such change brings performance improvement by reducing pressure on application to receive event per-CQE.

The proposed semantics follows the well-established kernel semantics.

* cq_max_count - defines the number of cookies needed to create an event.
* cq_period - defines the timeout between last event and a new one
that will occur even if cq_max_count was not reached

Quoted 6 lines

The patches are available in the git repository at:
git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git tags/rdma-next-2017-10-19-2

Thanks
---------------------------------------

Doug,

Please drop this series.

I'll resubmit it once user space part will be finished.

Mellanox,
I would like to use this opportunity and to ask publicly, please submit kernel and user spaces altogether.

Quoted 7 lines
Quoted 2 lines

It causes for too much errors and it is actually wrong to submit it separately.

Hi Leon,

I have a generic question. And I apologize if this is redundant and has been answered before. But what is the process for user-space submissions?

On top of Jason's answer. Sometimes, you can skip mailing list, usually it is spelling fixes or very local and small fixes to your code.

Quoted 7 lines

For libi40iw we make all submissions to this mailing list _only_ and this is all we are aware of. Are there alternate submission paths?

Shiraz

#21 lore raw
In reply to a message not in the archive
Quoted 3 lines · Leon Romanovsky

Mellanox,
I would like to use this opportunity and to ask publicly, please submit kernel and user spaces altogether.

+1

I was looking at the flow counters patches and they make no sense without also seeing the man pages that describe what the heck this stuff is.

At the very least put a rdma-core branch someplace and reference it in your kernel patch series, and it better have really good man pages for the verbs features being added.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20171023152638.GB11952-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org

author
#23 lore raw
Quoted 11 lines · Jason Gunthorpe
Quoted 3 lines · Leon Romanovsky

Mellanox,
I would like to use this opportunity and to ask publicly, please submit kernel and user spaces altogether.

+1

I was looking at the flow counters patches and they make no sense without also seeing the man pages that describe what the heck this stuff is.

+ kernel and user are one package, it makes no sense to split submission and to cause artificial delays between them.

Quoted 6 lines

At the very least put a rdma-core branch someplace and reference it in your kernel patch series, and it better have really good man pages for the verbs features being added.

Jason

#24 lore raw
In reply to a message not in the archive
Quoted 3 lines · Leon Romanovsky

Doug,

Please drop this series.

Done.

Signature · 8 lines
Doug Ledford <[email protected]>
    GPG KeyID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html