Skip to contentSkip to conversation

RDMA/efa: Add AH cache for AH reuse

PATCH Applied Yonatan Nachum 2 participants 4 messages Jul 6 – Jul 12

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v6 cover RDMA/efa: Add AH cache for AH reuse

Changelog:
v6:
* Remove rcu_barrier and kfree_rcu since all hashtable operations are serialized by the AH cache mutex, no concurrent readers to protect from.
v5: https://lore.kernel.org/all/[email protected]/
* Replace the single refcount and initialized flag with two counters:
one for entry lifetime in the hashtable and one for tracking active AH users.
v4: https://lore.kernel.org/all/[email protected]/
* Use kzalloc_obj for AH cache entry allocation instead of kzalloc
v3: https://lore.kernel.org/all/[email protected]/
* Address Sashiko comments in:
https://sashiko.dev/#/patchset/20260512061121.2177521-1-ynachum%40amazon.com
v2: https://lore.kernel.org/all/[email protected]/
* Zero-initialize AH cache key on cache lookup.
v1: https://lore.kernel.org/all/[email protected]/

-------------------------------------------------------------------------
New EFA devices don't support the creation of multiple AHs to the same remote on the same PD. To overcome this limitation, introduce an AH cache that manages AH reuse transparently.

The cache uses an rhashtable keyed by (PD, GID) to track active address handles with refcounts. On create AH, the driver returns an existing AH number if one is already cached, or creates a new one and caches it. On destroy AH, the driver only issues the device destroy command when the last reference is dropped.

A per-entry mutex serializes concurrent device commands on the same cache entry, preventing create-before-destroy races on the device.

The series was reviewed by Sashiko with Claude Opus 4.8.

Yonatan Nachum (2):
RDMA/efa: Add initialization of AH cache rhashtable RDMA/efa: Add AH cache handling on create and destroy AH

 drivers/infiniband/hw/efa/Makefile       |   4 +-
 drivers/infiniband/hw/efa/efa_ah_cache.c | 135 +++++++++++++++++++++++
 drivers/infiniband/hw/efa/efa_ah_cache.h |  39 +++++++
 drivers/infiniband/hw/efa/efa_com.c      |  12 +-
 drivers/infiniband/hw/efa/efa_com.h      |   3 +
 drivers/infiniband/hw/efa/efa_com_cmd.c  |  41 ++++++-
 drivers/infiniband/hw/efa/efa_com_cmd.h  |   1 +
 drivers/infiniband/hw/efa/efa_verbs.c    |   9 +-
 8 files changed, 233 insertions(+), 11 deletions(-)
 create mode 100644 drivers/infiniband/hw/efa/efa_ah_cache.c
 create mode 100644 drivers/infiniband/hw/efa/efa_ah_cache.h
Signature · 1 line
2.50.1
#1 lore raw

PATCH v6 1/2 RDMA/efa: Add initialization of AH cache rhashtable

New EFA devices don't support the creation of multiple address handles to the same remote on the same PD.
To overcome this limitation, introduce an AH cache rhashtable which will store the user refcounts of the same AH creation on the same PD and will allow the driver to manage AH reuse. The hashtable key is the combination of PD and GID. Add initialization and teardown logic for the rhashtable.

Each entry holds a refcount to manage the entry lifetime in the hashtable and a user count that indicates how many users are using the address handle.

---
 drivers/infiniband/hw/efa/Makefile       |  4 +--
 drivers/infiniband/hw/efa/efa_ah_cache.c | 41 ++++++++++++++++++++++++
 drivers/infiniband/hw/efa/efa_ah_cache.h | 36 +++++++++++++++++++++
 drivers/infiniband/hw/efa/efa_com.c      | 12 ++++++-
 drivers/infiniband/hw/efa/efa_com.h      |  3 ++
 5 files changed, 93 insertions(+), 3 deletions(-)
 create mode 100644 drivers/infiniband/hw/efa/efa_ah_cache.c
 create mode 100644 drivers/infiniband/hw/efa/efa_ah_cache.h
drivers/infiniband/hw/efa/Makefile +2 -2
index 6e83083af0bc..a6a433b0ba2f 100644
@@ -1,9 +1,9 @@
 # SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
-# Copyright 2018-2019 Amazon.com, Inc. or its affiliates. All rights reserved.
+# Copyright 2018-2026 Amazon.com, Inc. or its affiliates. All rights reserved.
 #
 # Makefile for Amazon Elastic Fabric Adapter (EFA) device driver.
 #
 
 obj-$(CONFIG_INFINIBAND_EFA) += efa.o
 
-efa-y := efa_com_cmd.o efa_com.o efa_main.o efa_verbs.o
+efa-y := efa_com_cmd.o efa_ah_cache.o efa_com.o efa_main.o efa_verbs.o
drivers/infiniband/hw/efa/efa_ah_cache.c new file +41 -0

41 lines not on this page · raw message

drivers/infiniband/hw/efa/efa_ah_cache.h new file +36 -0
new file mode 100644
index 000000000000..1d1fadb591cf
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * Copyright 2026 Amazon.com, Inc. or its affiliates. All rights reserved.
+ */
+
+#ifndef _EFA_AH_CACHE_H_
+#define _EFA_AH_CACHE_H_
+
+#include <linux/refcount.h>
+#include <linux/rhashtable.h>
+
+#define EFA_AH_GID_SIZE 16
+
+struct efa_ah_cache_key {
+	u8 gid[EFA_AH_GID_SIZE];
+	u16 pd;
+};
+
+struct efa_ah_cache_entry {
+	struct efa_ah_cache_key key;
+	u16 ah;
+	unsigned int usecnt;
+	refcount_t refcount;
+	struct rhash_head linkage;
+	struct mutex lock; /* Serializes device commands per cache entry */
+};
+
+struct efa_ah_cache {
+	struct rhashtable hashtable;
+	struct mutex lock; /* Protects AH cache hashtable */
+};
+
+int efa_ah_cache_init(struct efa_ah_cache *ah_cache);
+void efa_ah_cache_destroy(struct efa_ah_cache *ah_cache);
+
+#endif /* _EFA_AH_CACHE_H_ */
drivers/infiniband/hw/efa/efa_com.c +11 -1
index 7cc3f4af0bb9..7097d1c2f23d 100644
@@ -724,6 +724,8 @@ void efa_com_admin_destroy(struct efa_com_dev *edev)
 
 	size = aenq->depth * sizeof(*aenq->entries);
 	dma_free_coherent(edev->dmadev, size, aenq->entries, aenq->dma_addr);
+
+	efa_ah_cache_destroy(&edev->ah_cache);
 }
 
 /**
@@ -782,6 +784,12 @@ int efa_com_admin_init(struct efa_com_dev *edev,
 		return -ENODEV;
 	}
 
+	err = efa_ah_cache_init(&edev->ah_cache);
+	if (err) {
+		ibdev_err(edev->efa_dev, "Failed to init AH cache\n");
+		return err;
+	}
+
 	aq->depth = EFA_ADMIN_QUEUE_DEPTH;
 
 	aq->dmadev = edev->dmadev;
@@ -794,7 +802,7 @@ int efa_com_admin_init(struct efa_com_dev *edev,
 
 	err = efa_com_init_comp_ctxt(aq);
 	if (err)
-		return err;
+		goto err_destroy_ah_cache;
 
 	err = efa_com_admin_init_sq(edev);
 	if (err)
@@ -832,6 +840,8 @@ int efa_com_admin_init(struct efa_com_dev *edev,
 			  aq->sq.entries, aq->sq.dma_addr);
 err_destroy_comp_ctxt:
 	devm_kfree(edev->dmadev, aq->comp_ctx);
+err_destroy_ah_cache:
+	efa_ah_cache_destroy(&edev->ah_cache);
 
 	return err;
 }
drivers/infiniband/hw/efa/efa_com.h +3 -0
index f8c692b0e092..599db9d583bf 100644
@@ -14,6 +14,7 @@
 
 #include <rdma/ib_verbs.h>
 
+#include "efa_ah_cache.h"
 #include "efa_common_defs.h"
 #include "efa_admin_defs.h"
 #include "efa_admin_cmds_defs.h"
@@ -113,6 +114,8 @@ struct efa_com_dev {
 	u32 supported_features;
 	u32 dma_addr_bits;
 
+	struct efa_ah_cache ah_cache;
+
 	u32 dev_api_ver;
 	struct efa_com_mmio_read mmio_read;
 };
Signature · 1 line
2.50.1
#2 lore raw

PATCH v6 2/2 RDMA/efa: Add AH cache handling on create and destroy AH

In reply to Yonatan Nachum

On create AH, first check if the AH cache entry already exists and if so, returns the already stored AH number. If the entry doesn't exist, the driver creates it and calls the device to create the AH. A per-entry mutex serializes concurrent device commands on the same AH cache entry, ensuring only one thread issues the device create while others wait and reuse the result. If the device create fails, the entry's user count remains zero so subsequent threads will retry the device create.

On destroy AH, the user count is decremented under the entry mutex. If it reaches zero, the driver issues the device destroy command. After the device destroy completes, it removes the entry from the hashtable and frees it if no other references exist. If new users arrived during the destroy, the entry remains in the hashtable for reuse.

---
 drivers/infiniband/hw/efa/efa_ah_cache.c | 94 ++++++++++++++++++++++++
 drivers/infiniband/hw/efa/efa_ah_cache.h |  3 +
 drivers/infiniband/hw/efa/efa_com_cmd.c  | 41 ++++++++++-
 drivers/infiniband/hw/efa/efa_com_cmd.h  |  1 +
 drivers/infiniband/hw/efa/efa_verbs.c    |  9 ++-
 5 files changed, 140 insertions(+), 8 deletions(-)
drivers/infiniband/hw/efa/efa_ah_cache.c +94 -0
index 39d0d2e97589..6219529fa889 100644
@@ -39,3 +39,97 @@ void efa_ah_cache_destroy(struct efa_ah_cache *ah_cache)
 	rhashtable_free_and_destroy(&ah_cache->hashtable, efa_ah_cache_entry_free, NULL);
 	mutex_destroy(&ah_cache->lock);
 }
+
+static struct efa_ah_cache_entry *efa_ah_cache_lookup_locked(struct efa_ah_cache *ah_cache, u16 pd,
+							     u8 *gid)
+	__must_hold(&ah_cache->lock)
+{
+	struct efa_ah_cache_key key = {};
+
+	memcpy(key.gid, gid, sizeof(key.gid));
+	key.pd = pd;
+
+	return rhashtable_lookup_fast(&ah_cache->hashtable, &key, ah_cache_params);
+}
+
+struct efa_ah_cache_entry *efa_ah_cache_lookup(struct efa_ah_cache *ah_cache, u16 pd, u8 *gid)
+{
+	struct efa_ah_cache_entry *entry;
+
+	mutex_lock(&ah_cache->lock);
+	entry = efa_ah_cache_lookup_locked(ah_cache, pd, gid);
+	mutex_unlock(&ah_cache->lock);
+
+	return entry;
+}
+
+/**
+ * efa_ah_cache_get - Get or create an AH cache entry
+ * @ah_cache: AH cache
+ * @pd: Protection domain number
+ * @gid: GID address
+ *
+ * Look up an AH cache entry by PD and GID. If found, take a reference and
+ * return it. If not found, allocate a new entry and insert it. The caller must lock
+ * the entry mutex and check usecnt to determine whether a device create
+ * command is needed.
+ *
+ * Return: Pointer to the entry on success, ERR_PTR on failure.
+ */
+struct efa_ah_cache_entry *efa_ah_cache_get(struct efa_ah_cache *ah_cache, u16 pd, u8 *gid)
+{
+	struct efa_ah_cache_entry *entry;
+	int err;
+
+	mutex_lock(&ah_cache->lock);
+
+	entry = efa_ah_cache_lookup_locked(ah_cache, pd, gid);
+	if (entry) {
+		refcount_inc(&entry->refcount);
+		mutex_unlock(&ah_cache->lock);
+		return entry;
+	}
+
+	entry = kzalloc_obj(*entry);
+	if (!entry) {
+		mutex_unlock(&ah_cache->lock);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	memcpy(entry->key.gid, gid, sizeof(entry->key.gid));
+	entry->key.pd = pd;
+	refcount_set(&entry->refcount, 1);
+	mutex_init(&entry->lock);
+
+	err = rhashtable_insert_fast(&ah_cache->hashtable, &entry->linkage, ah_cache_params);
+	if (err) {
+		mutex_destroy(&entry->lock);
+		kfree(entry);
+		mutex_unlock(&ah_cache->lock);
+		return ERR_PTR(err);
+	}
+
+	mutex_unlock(&ah_cache->lock);
+	return entry;
+}
+
+/**
+ * efa_ah_cache_put - Put a refcount of an AH cache entry
+ * @ah_cache: AH cache
+ * @entry: AH cache entry
+ *
+ * Drop the refcount. If it reaches zero, remove the entry from the hashtable
+ * and free it.
+ */
+void efa_ah_cache_put(struct efa_ah_cache *ah_cache, struct efa_ah_cache_entry *entry)
+{
+	if (!refcount_dec_and_mutex_lock(&entry->refcount, &ah_cache->lock))
+		return;
+
+	/* AH cache lock is held here */
+	rhashtable_remove_fast(&ah_cache->hashtable, &entry->linkage, ah_cache_params);
+	mutex_unlock(&ah_cache->lock);
+
+	mutex_destroy(&entry->lock);
+	kfree(entry);
+}
drivers/infiniband/hw/efa/efa_ah_cache.h +3 -0
index 1d1fadb591cf..e7cdcbb64070 100644
@@ -32,5 +32,8 @@ struct efa_ah_cache {
 
 int efa_ah_cache_init(struct efa_ah_cache *ah_cache);
 void efa_ah_cache_destroy(struct efa_ah_cache *ah_cache);
+struct efa_ah_cache_entry *efa_ah_cache_get(struct efa_ah_cache *ah_cache, u16 pd, u8 *gid);
+struct efa_ah_cache_entry *efa_ah_cache_lookup(struct efa_ah_cache *ah_cache, u16 pd, u8 *gid);
+void efa_ah_cache_put(struct efa_ah_cache *ah_cache, struct efa_ah_cache_entry *entry);
 
 #endif /* _EFA_AH_CACHE_H_ */
drivers/infiniband/hw/efa/efa_com_cmd.c +37 -4

82 lines not on this page · raw message

drivers/infiniband/hw/efa/efa_com_cmd.h +1 -0
index ef15b3c38429..39bd4e06684a 100644
@@ -106,6 +106,7 @@ struct efa_com_create_ah_result {
 
 struct efa_com_destroy_ah_params {
 	u16 ah;
+	u8 gid[EFA_GID_SIZE];
 	u16 pdn;
 };
 
drivers/infiniband/hw/efa/efa_verbs.c +5 -4
index 06d3365aeb56..ec124fbda637 100644
@@ -2054,10 +2054,11 @@ int efa_mmap(struct ib_ucontext *ibucontext,
 
 static int efa_ah_destroy(struct efa_dev *dev, struct efa_ah *ah)
 {
-	struct efa_com_destroy_ah_params params = {
-		.ah = ah->ah,
-		.pdn = to_epd(ah->ibah.pd)->pdn,
-	};
+	struct efa_com_destroy_ah_params params = {};
+
+	params.ah = ah->ah;
+	memcpy(params.gid, ah->id, sizeof(params.gid));
+	params.pdn = to_epd(ah->ibah.pd)->pdn;
 
 	return efa_com_destroy_ah(&dev->edev, &params);
 }
Signature · 1 line
2.50.1
#3 lore raw
In reply to Yonatan Nachum
Quoted 12 lines · Yonatan Nachum

Changelog:
v6:
* Remove rcu_barrier and kfree_rcu since all hashtable operations are serialized by the AH cache mutex, no concurrent readers to protect from.
v5: https://lore.kernel.org/all/[email protected]/
* Replace the single refcount and initialized flag with two counters:
one for entry lifetime in the hashtable and one for tracking active AH users.
v4: https://lore.kernel.org/all/[email protected]/
* Use kzalloc_obj for AH cache entry allocation instead of kzalloc
v3: https://lore.kernel.org/all/[email protected]/
* Address Sashiko comments in:

Quoted 6 lines
   https://sashiko.dev/#/patchset/20260512061121.2177521-1-ynachum%40amazon.com

v2: https://lore.kernel.org/all/[email protected]/
* Zero-initialize AH cache key on cache lookup.
v1: https://lore.kernel.org/all/[email protected]/

[...]

Applied, thanks!

[1/2] RDMA/efa: Add initialization of AH cache rhashtable

      https://git.kernel.org/rdma/rdma/c/97ba15272e8e03
[2/2] RDMA/efa: Add AH cache handling on create and destroy AH
      https://git.kernel.org/rdma/rdma/c/234895fa8be06e

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>