Skip to content

RDMA/efa: Add AH cache for AH reuse

Yonatan Nachum linux-rdma 6 versions – Applied

Status from this message: Applied, thanks!

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v6 Latest2 patches 1 replyApplied
  2. v52 patches 5 repliesSuperseded
  3. v42 patches 6 repliesSuperseded
Earlier versions · 3
VersionPostedPatchesRepliesStatus
v32 patches1 replySuperseded
v22 patches12 repliesSuperseded
v12 patches1 replySuperseded

Changes in v6 from the cover letter

* Remove rcu_barrier and kfree_rcu since all hashtable operations are serialized by the AH cache mutex, no concurrent readers to protect from.
v5: https://lore.kernel.org/all/[email protected]/
* Replace the single refcount and initialized flag with two counters:
one for entry lifetime in the hashtable and one for tracking active AH users.
v4: https://lore.kernel.org/all/[email protected]/
* Use kzalloc_obj for AH cache entry allocation instead of kzalloc
v3: https://lore.kernel.org/all/[email protected]/
* Address Sashiko comments in:
https://sashiko.dev/#/patchset/20260512061121.2177521-1-ynachum%40amazon.com
v2: https://lore.kernel.org/all/[email protected]/
* Zero-initialize AH cache key on cache lookup.
v1: https://lore.kernel.org/all/[email protected]/

Patches in v6 Applied

#Subject and filesSizeRepliesApplied
1/2RDMA/efa: Add initialization of AH cache rhashtabledrivers/infiniband/hw/efa/Makefile drivers/infiniband/hw/efa/efa_ah_cache.c drivers/infiniband/hw/efa/efa_ah_cache.h +2 more+93 -30 replies
2/2RDMA/efa: Add AH cache handling on create and destroy AHdrivers/infiniband/hw/efa/efa_ah_cache.c drivers/infiniband/hw/efa/efa_ah_cache.h drivers/infiniband/hw/efa/efa_com_cmd.c +2 more+140 -80 replies

Reviews trailers per patch

All 2 patches have a Reviewed-by or Acked-by.

PatchFiras JahjahMichael Margolin
1/2R-b R-b
2/2R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v6 thread

Cover letter [PATCH for-next v6 0/2] RDMA/efa: Add AH cache for AH reuse

Changelog:
v6:
* Remove rcu_barrier and kfree_rcu since all hashtable operations are serialized by the AH cache mutex, no concurrent readers to protect from.
v5: https://lore.kernel.org/all/[email protected]/
* Replace the single refcount and initialized flag with two counters:
one for entry lifetime in the hashtable and one for tracking active AH users.
v4: https://lore.kernel.org/all/[email protected]/
* Use kzalloc_obj for AH cache entry allocation instead of kzalloc
v3: https://lore.kernel.org/all/[email protected]/
* Address Sashiko comments in:
https://sashiko.dev/#/patchset/20260512061121.2177521-1-ynachum%40amazon.com
v2: https://lore.kernel.org/all/[email protected]/
* Zero-initialize AH cache key on cache lookup.
v1: https://lore.kernel.org/all/[email protected]/

-------------------------------------------------------------------------
New EFA devices don't support the creation of multiple AHs to the same remote on the same PD. To overcome this limitation, introduce an AH cache that manages AH reuse transparently.

The cache uses an rhashtable keyed by (PD, GID) to track active address handles with refcounts. On create AH, the driver returns an existing AH number if one is already cached, or creates a new one and caches it. On destroy AH, the driver only issues the device destroy command when the last reference is dropped.

A per-entry mutex serializes concurrent device commands on the same cache entry, preventing create-before-destroy races on the device.

The series was reviewed by Sashiko with Claude Opus 4.8.

Yonatan Nachum (2):
RDMA/efa: Add initialization of AH cache rhashtable RDMA/efa: Add AH cache handling on create and destroy AH

 drivers/infiniband/hw/efa/Makefile       |   4 +-
 drivers/infiniband/hw/efa/efa_ah_cache.c | 135 +++++++++++++++++++++++
 drivers/infiniband/hw/efa/efa_ah_cache.h |  39 +++++++
 drivers/infiniband/hw/efa/efa_com.c      |  12 +-
 drivers/infiniband/hw/efa/efa_com.h      |   3 +
 drivers/infiniband/hw/efa/efa_com_cmd.c  |  41 ++++++-
 drivers/infiniband/hw/efa/efa_com_cmd.h  |   1 +
 drivers/infiniband/hw/efa/efa_verbs.c    |   9 +-
 8 files changed, 233 insertions(+), 11 deletions(-)
 create mode 100644 drivers/infiniband/hw/efa/efa_ah_cache.c
 create mode 100644 drivers/infiniband/hw/efa/efa_ah_cache.h
Signature · 1 line
2.50.1