Skip to content

RDMA resource tracking

Leon Romanovsky linux-rdma netdev 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest6 of 9 patches 3 repliesIn review

6 of 9 patches are in this archive. The cover letter announces 9.

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
3/9rdma: Add filtering infrastructurerdma/rdma.c rdma/rdma.h rdma/utils.c+228 -01 reply
4/9rdma: Set pointer to device name positionrdma/rdma.h rdma/utils.c+20 -00 replies
5/9rdma: Allow external usage of compare string routinerdma/rdma.h rdma/utils.c+3 -10 replies
7/9rdma: Add resource tracking summaryrdma/Makefile rdma/rdma.c rdma/rdma.h +2 more+177 -20 replies
8/9rdma: Add QP resource tracking informationrdma/res.c rdma/utils.c+254 -01 reply
9/9rdma: Document resource trackingman/man8/rdma-resource.8+91 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH iproute2-next 0/9] RDMA resource tracking

From: Leon Romanovsky <[email protected]>

Hi,

This is supplementary (user-space) part of RDMA resource tracking posted to the RDMA mailing list for the review [1].

The main goal of this new functionality in RDMAtool is to provide debug visibility of running applications in RDMA stack.

The current series adds new command object (resource) which provides short summary if it is called without arguments or more detailed information while it is called with request to present QPs.

1) Summary information:

$ rdma res
1: mlx5_0: curr/max: pd 3/16777216 cq 5/16777216 qp 4/262144
2: mlx5_1: curr/max: pd 3/16777216 cq 5/16777216 qp 4/262144
3: mlx5_2: curr/max: pd 3/16777216 cq 5/16777216 qp 4/262144
4: mlx5_3: curr/max: pd 2/16777216 cq 3/16777216 qp 2/262144
5: mlx5_4: curr/max: pd 3/16777216 cq 5/16777216 qp 4/262144

2) Summary information of specific device:

$ rdma res show mlx5_4
5: mlx5_4: curr/max: pd 3/16777216 cq 5/16777216 qp 4/262144

3) Detailed information of specific device:

$ rdma res show qp link mlx5_4
DEV/PORT  LQPN       TYPE  STATE  PID        COMM
mlx5_4/-  8          UD    RESET  0          [ipoib-verbs]
mlx5_4/1  7          UD    RTS    0          [mlx5-gsi]
mlx5_4/1  1          GSI   RTS    0          [rdma-mad]
mlx5_4/1  0          SMI   RTS    0          [rdma-mad]
$ rdma res show qp link mlx5_4/
DEV/PORT  LQPN       TYPE  STATE  PID        COMM
mlx5_4/-  8          UD    RESET  0          [ipoib-verbs]
mlx5_4/1  7          UD    RTS    0          [mlx5-gsi]
mlx5_4/1  1          GSI   RTS    0          [rdma-mad]
mlx5_4/1  0          SMI   RTS    0          [rdma-mad]

4) Wrong port (it can be 1 or 2):

$ rdma res show qp link mlx5_4/0
Wrong device name

5) Detailed information of specific device and port:

$ rdma res show qp link mlx5_4/1
DEV/PORT  LQPN       TYPE  STATE  PID        COMM
mlx5_4/1  7          UD    RTS    0          [mlx5-gsi]
mlx5_4/1  1          GSI   RTS    0          [rdma-mad]
mlx5_4/1  0          SMI   RTS    0          [rdma-mad]

6) Detailed information of QPs not-connected to port yet:

$ rdma res show qp link mlx5_4/-
DEV/PORT  LQPN       TYPE  STATE  PID        COMM
mlx5_4/-  8          UD    RESET  0          [ipoib-verbs]

7) Very detailed view:

$ rdma res show qp link mlx5_4/- -d
DEV/PORT  LQPN       RQPN       TYPE  STATE  PID        COMM            SQ-PSN     RQ-PSN     PATH-MIG
mlx5_4/-  8          ---        UD    RESET  0          [ipoib-verbs]   0          ---        ---

8) Limit display to specific columns:

$ rdma res show qp link mlx5_4/1 display pid,lqpn,comm
DEV/PORT  LQPN       PID        COMM
mlx5_4/1  7          0          [mlx5-gsi]
mlx5_4/1  1          0          [rdma-mad]
mlx5_4/1  0          0          [rdma-mad]

9) Filter specific processes:

$ rdma res show qp link mlx5_4/1 display pid,lqpn,comm pid 0-2000
DEV/PORT  LQPN       PID        COMM
mlx5_4/1  7          0          [mlx5-gsi]
mlx5_4/1  1          0          [rdma-mad]
mlx5_4/1  0          0          [rdma-mad]

Thanks

[1] https://marc.info/?l=linux-rdma&m=151488112032562&w=2

Leon Romanovsky (9):
rdam: Add option to provide "-" sign for the port number
rdma: Make visible the number of arguments
rdma: Add filtering infrastructure
rdma: Set pointer to device name position
rdma: Allow external usage of compare string routine
rdma: Update kernel header file
rdma: Add resource tracking summary
rdma: Add QP resource tracking information
rdma: Document resource tracking

 include/uapi/rdma/rdma_netlink.h |  58 +++++-
 man/man8/rdma-resource.8         |  91 +++++++++
 rdma/Makefile                    |   2 +-
 rdma/link.c                      |   2 +-
 rdma/rdma.c                      |   4 +-
 rdma/rdma.h                      |  23 ++-
 rdma/res.c                       | 411 +++++++++++++++++++++++++++++++++++++++
 rdma/utils.c                     | 300 +++++++++++++++++++++++++++-
 8 files changed, 874 insertions(+), 17 deletions(-)
 create mode 100644 man/man8/rdma-resource.8
 create mode 100644 rdma/res.c