RDMA resource tracking

Changelog:
v6->v7:
* Added completion mechanism for rdam_restrack_put()
* Converted init funciton to be void
* Changed to use kref_get_unless_zero()
* Forced valid to be false for not-added QPs.
* Added centralized place to update index for skipped QPs.
v5->v6:
* Removed SRCU
* Added kref logic to protect resources
* Dropped support of XRC QPs for now
v4->v5:
* Rewrote logic to acquire kernel owners of PD and CQ objects
* Simplified interface.
* Added logic to return summary for all PID namespaces and not only init PID namespace.
* Changed DB implementation from struct with arrays to be struct with hash table.
* Moved to use SRCU per-hash table.
* Dropped RW semaphores in favor of mutex to protect linked lists.
* Dropped QP named annotation patch.
* Used SPDX licenses.
v3->v4:
* Added Steve ROB tags.
* Updated restrack.h to be compatible with kernel-doc format
* Fixed bug when not all QPs (more than one netlink page) were returned by netlink interface
* Removed "MAX" values from object summary. In followup patches, I will eturn object summary per-namespace and not for init PID namespace only. Max values can be seen with ibv_devinfo tool.
* Renamed "comm" to be kern_name.
* Fix spelling errors.
v2->v3:
* Added support of PID namespaces.
* Rewrote rdma_restraack_add function to ensure that it won't appear in lists before it is valid.
* Replace pid/task name caching logic to use task_struct instead.
* Removed report of name of task's for user space objects. Users are expected to read it through /proc/PID/comm.
v1->v2:
* Rebased on latest rdma/for-next
* Replaced mutex lock which protect linked lists to be RW semaphore. It has no impact on current implementation, because there is only one reader (nldev) and it is serialized. However better to be prepared for multiple readers from the beginning.
* Added reinitialization next QP entry logic to ensure that it exists and didn't vanish during fill_req_qp() work.
v0->v1:
* Dropped RFC
* Separated to thre series, one for for-rc, and two for-next.

-------
The original goal of this series was to allow ability to view connection (QP) information about running processes, however I used this opportunity and created common infrastructure to track and report various resources. The report part is implemented in netlink (nldev), but smart ULPs can now create advanced usage models based on device utilization.

The current implementation relies on one lock per-object per-device, so creation/destroying of various objects (CQ, PD, e.t.c) on various or the same devices doesn't interfere each with another.

The data protection is performed with SRCU and its reader-writer model ensures that resource won't be destroyed till readers will finish their work.

Possible future work will include:
* Reducing number of locks in RDMA, because of SRCU.
* Converting CMA to be based completely on resource tracking.
* Addition of other objects and extending current to give full and detailed state of the RDMA kernel stack.
* Replacing synchronize_srcu with call_srcu to make destroy flow non-blocking.
* Provide reliable device reset flow, preserving resource creation ordering.

Thanks
---------------------------------------

Leon Romanovsky (8):
RDMA/core: Print caller name instead of function name RDMA/core: Save kernel caller name in PD and CQ objects RDMA/restrack: Add general infrastructure to track RDMA resources RDMA/core: Add resource tracking for create and destroy QPs RDMA/core: Add resource tracking for create and destroy CQs RDMA/core: Add resource tracking for create and destroy PDs RDMA/nldev: Provide global resource utilization RDMA/nldev: Provide detailed QP information

 drivers/infiniband/core/Makefile           |   2 +-
 drivers/infiniband/core/core_priv.h        |  28 +++
 drivers/infiniband/core/cq.c               |  16 +-
 drivers/infiniband/core/device.c           |   4 +
 drivers/infiniband/core/nldev.c            | 378 +++++++++++++++++++++++++++++
 drivers/infiniband/core/restrack.c         | 166 +++++++++++++
 drivers/infiniband/core/uverbs_cmd.c       |   7 +-
 drivers/infiniband/core/uverbs_std_types.c |   3 +
 drivers/infiniband/core/verbs.c            |  18 +-
 include/rdma/ib_verbs.h                    |  34 ++-
 include/rdma/restrack.h                    | 157 ++++++++++++
 include/uapi/rdma/rdma_netlink.h           |  55 +++++
 12 files changed, 851 insertions(+), 17 deletions(-)
 create mode 100644 drivers/infiniband/core/restrack.c
 create mode 100644 include/rdma/restrack.h

--
2.16.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html