Global per-type support for QP counters

PATCH New Leon Romanovsky 1 participant 4 messages Jul 2, 2020 InfiniBand

lore

Share & tools

lore

Leon Romanovsky

author

2020-07-02 08:29 UTC #1 lore raw

PATCH 1/3 RDMA/counter: Add PID category support in auto mode

From: Mark Zhang <[email protected]>

With the "PID" category QPs have same PID will be bound to same counter; If this category is not set then QPs have different PIDs will be bound to same counter.

This is implemented for 2 reasons:
1. The counter is a limited resource, while there may be dozens of applications, each of which creates several types of QPs, which means it may doesn't have enough counter.
2. The system administrator needs all QPs created by all applications with same type bound to one counter.

The counter name and PID is only make sense when "PID" category are configured.

This category can also be used in combine with others, e.g. QP type.

Signed-off-by Mark Zhang <[email protected]>

Reviewed-by Maor Gottlieb <[email protected]>

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/infiniband/core/counters.c | 20 +++++---------------
 drivers/infiniband/core/nldev.c    |  8 ++++++--
 include/uapi/rdma/rdma_netlink.h   |  1 +
 3 files changed, 12 insertions(+), 17 deletions(-)

drivers/infiniband/core/counters.c +5 ~~-15~~

index 738d1faf4bba..9ec8631c3970 100644
@@ -8,7 +8,7 @@
 #include "core_priv.h"
 #include "restrack.h"
 
-#define ALL_AUTO_MODE_MASKS (RDMA_COUNTER_MASK_QP_TYPE)
+#define ALL_AUTO_MODE_MASKS (RDMA_COUNTER_MASK_QP_TYPE | RDMA_COUNTER_MASK_PID)
 
 static int __counter_set_mode(struct rdma_counter_mode *curr,
 			      enum rdma_nl_counter_mode new_mode,

@@ -149,23 +149,13 @@ static bool auto_mode_match(struct ib_qp *qp, struct rdma_counter *counter,
 	struct auto_mode_param *param = &counter->mode.param;
 	bool match = true;
 
-	/*
-	 * Ensure that counter belongs to the right PID.  This operation can
-	 * race with user space which kills the process and leaves QP and
-	 * counters orphans.
-	 *
-	 * It is not a big deal because exitted task will leave both QP and
-	 * counter in the same bucket of zombie process. Just ensure that
-	 * process is still alive before procedding.
-	 *
-	 */
-	if (task_pid_nr(counter->res.task) != task_pid_nr(qp->res.task) ||
-	    !task_pid_nr(qp->res.task))
-		return false;
-
 	if (auto_mask & RDMA_COUNTER_MASK_QP_TYPE)
 		match &= (param->qp_type == qp->qp_type);
 
+	if (auto_mask & RDMA_COUNTER_MASK_PID)
+		match &= (task_pid_nr(counter->res.task) ==
+			  task_pid_nr(qp->res.task));
+
 	return match;
 }

drivers/infiniband/core/nldev.c +6 -2

index 1051b5622b62..76af7ea2875d 100644
@@ -711,11 +711,16 @@ static int fill_stat_counter_mode(struct sk_buff *msg,
 	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_STAT_MODE, m->mode))
 		return -EMSGSIZE;
 
-	if (m->mode == RDMA_COUNTER_MODE_AUTO)
+	if (m->mode == RDMA_COUNTER_MODE_AUTO) {
 		if ((m->mask & RDMA_COUNTER_MASK_QP_TYPE) &&
 		    nla_put_u8(msg, RDMA_NLDEV_ATTR_RES_TYPE, m->param.qp_type))
 			return -EMSGSIZE;
 
+		if ((m->mask & RDMA_COUNTER_MASK_PID) &&
+		    fill_res_name_pid(msg, &counter->res))
+			return -EMSGSIZE;
+	}
+
 	return 0;
 }

@@ -855,7 +860,6 @@ static int fill_res_counter_entry(struct sk_buff *msg, bool has_cap_net_admin,
 
 	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_PORT_INDEX, counter->port) ||
 	    nla_put_u32(msg, RDMA_NLDEV_ATTR_STAT_COUNTER_ID, counter->id) ||
-	    fill_res_name_pid(msg, &counter->res) ||
 	    fill_stat_counter_mode(msg, counter) ||
 	    fill_stat_counter_qps(msg, counter) ||
 	    fill_stat_counter_hwcounters(msg, counter))

include/uapi/rdma/rdma_netlink.h +1 -0

index 3826143d420d..d2f5b8396243 100644
@@ -569,5 +569,6 @@ enum rdma_nl_counter_mode {
  */
 enum rdma_nl_counter_mask {
 	RDMA_COUNTER_MASK_QP_TYPE = 1,
+	RDMA_COUNTER_MASK_PID = 1 << 1,
 };
 #endif /* _UAPI_RDMA_NETLINK_H */

Signature · 1 line

2.26.2

Leon Romanovsky

author

2020-07-02 08:29 UTC #2 lore raw

PATCH 2/3 RDMA/counter: Only bind user QPs in auto mode

In reply to Leon Romanovsky

From: Mark Zhang <[email protected]>

In auto mode only bind user QPs to a dynamic counter, since this feature is mainly used for system statistic and diagnostic purpose, while there's no need to counter kernel QPs so far.

Fixes 99fa331dc862 ("RDMA/counter: Add "auto" configuration mode support")

Signed-off-by Mark Zhang <[email protected]>

Reviewed-by Maor Gottlieb <[email protected]>

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/infiniband/core/counters.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

drivers/infiniband/core/counters.c +1 -1

index 9ec8631c3970..104df52e9bdb 100644
@@ -278,7 +278,7 @@ int rdma_counter_bind_qp_auto(struct ib_qp *qp, u8 port)
 	struct rdma_counter *counter;
 	int ret;
 
-	if (!qp->res.valid)
+	if (!qp->res.valid || rdma_is_kernel_res(&qp->res))
 		return 0;
 
 	if (!rdma_is_port_valid(dev, port))

Signature · 1 line

2.26.2

Leon Romanovsky

author

2020-07-02 08:29 UTC #3 lore raw

PATCH 3/3 RDMA/counter: Allow manually bind QPs with different pids to same counter

In reply to Leon Romanovsky

From: Mark Zhang <[email protected]>

In manual mode allow bind user QPs with different pids to same counter, since this is allowed in auto mode.
Bind kernel QPs and user QPs to the same counter are not allowed.

Fixes 1bd8e0a9d0fd ("RDMA/counter: Allow manual mode configuration support")

Signed-off-by Mark Zhang <[email protected]>

Reviewed-by Maor Gottlieb <[email protected]>

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/infiniband/core/counters.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

drivers/infiniband/core/counters.c +1 -1

index 104df52e9bdb..636166880442 100644
@@ -473,7 +473,7 @@ int rdma_counter_bind_qpn(struct ib_device *dev, u8 port,
 		goto err;
 	}
 
-	if (counter->res.task != qp->res.task) {
+	if (rdma_is_kernel_res(&counter->res) != rdma_is_kernel_res(&qp->res)) {
 		ret = -EINVAL;
 		goto err_task;
 	}

Signature · 1 line

2.26.2