Skip to contentSkip to conversation

RDMA/efa: Cleanups and minor improvements

PATCH In review Michael Margolin 4 participants 19 messages Jun 24, 2024 – Jun 28, 2024

Subsystems (2)
lore
Share & tools
lore

Michael Margolin

author maintainer
#0 lore raw

cover RDMA/efa: Cleanups and minor improvements

This series contains few cleanups and improvements that were laying around.

Gal Pressman (1):
RDMA/efa: Use offset_in_page() function

Michael Margolin (2):
RDMA/efa: Move type conversion helpers to efa.h RDMA/efa: Align private func names to a single convention

Yonatan Nachum (2):
RDMA/efa: Remove duplicate aenq enable macro
RDMA/efa: Validate EQ array out of bounds reach

 drivers/infiniband/hw/efa/efa.h       |  41 +++-
 drivers/infiniband/hw/efa/efa_main.c  |   2 +-
 drivers/infiniband/hw/efa/efa_verbs.c | 333 +++++++++++---------------
 3 files changed, 174 insertions(+), 202 deletions(-)
Signature · 1 line
2.40.1
author maintainer
#1 lore raw

PATCH 1/5 RDMA/efa: Use offset_in_page() function

From: Gal Pressman <[email protected]>

Use offset_in_page() instead of open-coding it.

---
 drivers/infiniband/hw/efa/efa_verbs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/hw/efa/efa_verbs.c +1 -1
index 8f7a13b79cdc..eee2ae215414 100644
@@ -524,7 +524,7 @@ static int qp_mmap_entries_setup(struct efa_qp *qp,
 
 	address = dev->mem_bar_addr + resp->llq_desc_offset;
 	length = PAGE_ALIGN(params->sq_ring_size_in_bytes +
-			    (resp->llq_desc_offset & ~PAGE_MASK));
+			    offset_in_page(resp->llq_desc_offset));
 
 	qp->llq_desc_mmap_entry =
 		efa_user_mmap_entry_insert(&ucontext->ibucontext,
Signature · 1 line
2.40.1
author maintainer
#3 lore raw

PATCH 2/5 RDMA/efa: Remove duplicate aenq enable macro

In reply to Michael Margolin

From: Yonatan Nachum <[email protected]>

We have the same macro in main and verbs files and we don't use the macro in the verbs file, remove it.

---
 drivers/infiniband/hw/efa/efa_verbs.c | 4 ----
 1 file changed, 4 deletions(-)
drivers/infiniband/hw/efa/efa_verbs.c +0 -4
index eee2ae215414..cd1f735d08a7 100644
@@ -26,10 +26,6 @@ enum {
 	EFA_MMAP_IO_NC,
 };
 
-#define EFA_AENQ_ENABLED_GROUPS \
-	(BIT(EFA_ADMIN_FATAL_ERROR) | BIT(EFA_ADMIN_WARNING) | \
-	 BIT(EFA_ADMIN_NOTIFICATION) | BIT(EFA_ADMIN_KEEP_ALIVE))
-
 struct efa_user_mmap_entry {
 	struct rdma_user_mmap_entry rdma_entry;
 	u64 address;
Signature · 1 line
2.40.1
reviewer Reviewed-by on v1 2/5
#4 lore raw

Re: [PATCH for-next 2/5] RDMA/efa: Remove duplicate aenq enable macro

Quoted 8 lines · Michael Margolin

From: Yonatan Nachum <[email protected]>

We have the same macro in main and verbs files and we don't use the macro in the verbs file, remove it.

author maintainer
#5 lore raw

PATCH 3/5 RDMA/efa: Validate EQ array out of bounds reach

In reply to Michael Margolin

From: Yonatan Nachum <[email protected]>

When creating a new CQ with interrupts enabled, the caller needs to specify an EQ index to which the interrupts will be sent on, we don't validate the requested index in the EQ array.
Validate out of bound reach of the EQ array and return an error.

This is not a bug because IB core validates the requested EQ number when creating a CQ.

---
 drivers/infiniband/hw/efa/efa_verbs.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/infiniband/hw/efa/efa_verbs.c +6 -1
index cd1f735d08a7..9c3e476e3f9c 100644
@@ -1046,7 +1046,7 @@ int efa_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata)
 
 static struct efa_eq *efa_vec2eq(struct efa_dev *dev, int vec)
 {
-	return &dev->eqs[vec];
+	return vec < dev->neqs ? &dev->eqs[vec] : NULL;
 }
 
 static int cq_mmap_entries_setup(struct efa_dev *dev, struct efa_cq *cq,
@@ -1173,6 +1173,11 @@ int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 	params.set_src_addr = set_src_addr;
 	if (cmd.flags & EFA_CREATE_CQ_WITH_COMPLETION_CHANNEL) {
 		cq->eq = efa_vec2eq(dev, attr->comp_vector);
+		if (!cq->eq) {
+			ibdev_dbg(ibdev, "Invalid EQ requested[%u]\n", attr->comp_vector);
+			err = -EINVAL;
+			goto err_free_mapped;
+		}
 		params.eqn = cq->eq->eeq.eqn;
 		params.interrupt_mode_enabled = true;
 	}
Signature · 1 line
2.40.1
reviewer
#6 lore raw

Re: [PATCH for-next 3/5] RDMA/efa: Validate EQ array out of bounds reach

Quoted 9 lines · Michael Margolin

From: Yonatan Nachum <[email protected]>

When creating a new CQ with interrupts enabled, the caller needs to specify an EQ index to which the interrupts will be sent on, we don't validate the requested index in the EQ array.
Validate out of bound reach of the EQ array and return an error.

This is not a bug because IB core validates the requested EQ number when creating a CQ.

Then why is this patch needed?

Quoted 4 lines
author maintainer
#7 lore raw

Re: [PATCH for-next 3/5] RDMA/efa: Validate EQ array out of bounds reach

Quoted 7 lines · Gal Pressman

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

Quoted 2 lines · Michael Margolin

From: Yonatan Nachum <[email protected]>

Quoted 8 lines
Quoted 7 lines

When creating a new CQ with interrupts enabled, the caller needs to specify an EQ index to which the interrupts will be sent on, we don't validate the requested index in the EQ array.
Validate out of bound reach of the EQ array and return an error.

This is not a bug because IB core validates the requested EQ number when creating a CQ.

Then why is this patch needed?

Mainly as best practice of validating the index right before array access to avoid hard to catch bugs.

Michael

#8 lore raw

Re: [PATCH for-next 3/5] RDMA/efa: Validate EQ array out of bounds reach

Quoted 14 lines · Michael
Quoted 12 lines · Gal Pressman

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

Quoted 7 lines · Michael Margolin

From: Yonatan Nachum <[email protected]>

When creating a new CQ with interrupts enabled, the caller needs to specify an EQ index to which the interrupts will be sent on, we don't validate the requested index in the EQ array.
Validate out of bound reach of the EQ array and return an error.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

This is not a bug because IB core validates the requested EQ number when creating a CQ.

Then why is this patch needed?

Mainly as best practice of validating the index right before array access to avoid hard to catch bugs.

If you know about the bug, please fix it. There is no need to add code "to protect" low level from incorrect upper level.

Thanks

Quoted 4 lines

Michael

author maintainer
#9 lore raw

PATCH 4/5 RDMA/efa: Move type conversion helpers to efa.h

In reply to Michael Margolin

Move ib_ to efa_ types conversion functions to have them near the types definitions and to reduce code in efa_verbs.c.

---
 drivers/infiniband/hw/efa/efa.h       | 35 +++++++++++++++++++++++++++
 drivers/infiniband/hw/efa/efa_verbs.c | 35 ---------------------------
 2 files changed, 35 insertions(+), 35 deletions(-)
drivers/infiniband/hw/efa/efa.h +35 -0
index 926f9ff1f60f..f1020bf85b78 100644
@@ -142,6 +142,41 @@ struct efa_eq {
 	struct efa_irq irq;
 };
 
+static inline struct efa_dev *to_edev(struct ib_device *ibdev)
+{
+	return container_of(ibdev, struct efa_dev, ibdev);
+}
+
+static inline struct efa_ucontext *to_eucontext(struct ib_ucontext *ibucontext)
+{
+	return container_of(ibucontext, struct efa_ucontext, ibucontext);
+}
+
+static inline struct efa_pd *to_epd(struct ib_pd *ibpd)
+{
+	return container_of(ibpd, struct efa_pd, ibpd);
+}
+
+static inline struct efa_mr *to_emr(struct ib_mr *ibmr)
+{
+	return container_of(ibmr, struct efa_mr, ibmr);
+}
+
+static inline struct efa_qp *to_eqp(struct ib_qp *ibqp)
+{
+	return container_of(ibqp, struct efa_qp, ibqp);
+}
+
+static inline struct efa_cq *to_ecq(struct ib_cq *ibcq)
+{
+	return container_of(ibcq, struct efa_cq, ibcq);
+}
+
+static inline struct efa_ah *to_eah(struct ib_ah *ibah)
+{
+	return container_of(ibah, struct efa_ah, ibah);
+}
+
 int efa_query_device(struct ib_device *ibdev,
 		     struct ib_device_attr *props,
 		     struct ib_udata *udata);
drivers/infiniband/hw/efa/efa_verbs.c +0 -35
index 9c3e476e3f9c..34a9f86af9bd 100644
@@ -127,41 +127,6 @@ struct pbl_context {
 	u8 physically_continuous;
 };
 
-static inline struct efa_dev *to_edev(struct ib_device *ibdev)
-{
-	return container_of(ibdev, struct efa_dev, ibdev);
-}
-
-static inline struct efa_ucontext *to_eucontext(struct ib_ucontext *ibucontext)
-{
-	return container_of(ibucontext, struct efa_ucontext, ibucontext);
-}
-
-static inline struct efa_pd *to_epd(struct ib_pd *ibpd)
-{
-	return container_of(ibpd, struct efa_pd, ibpd);
-}
-
-static inline struct efa_mr *to_emr(struct ib_mr *ibmr)
-{
-	return container_of(ibmr, struct efa_mr, ibmr);
-}
-
-static inline struct efa_qp *to_eqp(struct ib_qp *ibqp)
-{
-	return container_of(ibqp, struct efa_qp, ibqp);
-}
-
-static inline struct efa_cq *to_ecq(struct ib_cq *ibcq)
-{
-	return container_of(ibcq, struct efa_cq, ibcq);
-}
-
-static inline struct efa_ah *to_eah(struct ib_ah *ibah)
-{
-	return container_of(ibah, struct efa_ah, ibah);
-}
-
 static inline struct efa_user_mmap_entry *
 to_emmap(struct rdma_user_mmap_entry *rdma_entry)
 {
Signature · 1 line
2.40.1
reviewer
#10 lore raw

Re: [PATCH for-next 4/5] RDMA/efa: Move type conversion helpers to efa.h

Quoted 6 lines · Michael Margolin

Move ib_ to efa_ types conversion functions to have them near the types definitions and to reduce code in efa_verbs.c.

The idea was to not expose these functions in the h file as they are not used outside of efa_verbs.c.

author maintainer
#11 lore raw

Re: [PATCH for-next 4/5] RDMA/efa: Move type conversion helpers to efa.h

On 6/25/2024 9:33 AM, Gal Pressman wrote

Quoted 9 lines · Michael Margolin
Quoted 6 lines

Move ib_ to efa_ types conversion functions to have them near the types definitions and to reduce code in efa_verbs.c.

The idea was to not expose these functions in the h file as they are not used outside of efa_verbs.c.

I know but it's a driver internal header and it also makes sense to have the cast from the base type near the definition.

Michael

#12 lore raw

Re: [PATCH for-next 4/5] RDMA/efa: Move type conversion helpers to efa.h

Quoted 6 lines · Michael

On 6/25/2024 9:33 AM, Gal Pressman wrote

Quoted 4 lines · Michael Margolin
Quoted 3 lines

Move ib_ to efa_ types conversion functions to have them near the types definitions and to reduce code in efa_verbs.c.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

The idea was to not expose these functions in the h file as they are not used outside of efa_verbs.c.

I know but it's a driver internal header and it also makes sense to have the cast from the base type near the definition.

We put in header file things which are used in multiple files. It is not the case here. Let's try to avoid code churn without real gain behind it.

Thanks

Quoted 3 lines

Michael

author maintainer
#13 lore raw

PATCH 5/5 RDMA/efa: Align private func names to a single convention

In reply to Michael Margolin

Name functions that aren't exposed outside of a file with '_' prefix and make sure that all verbs ops' implementations are named according to the op name with an additional 'efa_' prefix.

---
 drivers/infiniband/hw/efa/efa.h       |   6 +-
 drivers/infiniband/hw/efa/efa_main.c  |   2 +-
 drivers/infiniband/hw/efa/efa_verbs.c | 285 ++++++++++++--------------
 3 files changed, 132 insertions(+), 161 deletions(-)
drivers/infiniband/hw/efa/efa.h +3 -3
index f1020bf85b78..655db23e81b6 100644
@@ -197,9 +197,9 @@ int efa_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *init_attr,
 int efa_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata);
 int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 		  struct ib_udata *udata);
-struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
-			 u64 virt_addr, int access_flags,
-			 struct ib_udata *udata);
+struct ib_mr *efa_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 length,
+			      u64 virt_addr, int access_flags,
+			      struct ib_udata *udata);
 struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start,
 				     u64 length, u64 virt_addr,
 				     int fd, int access_flags,
drivers/infiniband/hw/efa/efa_main.c +1 -1
index 99f9ac23c721..bff2485a76a0 100644
@@ -394,7 +394,7 @@ static const struct ib_device_ops efa_dev_ops = {
 	.query_pkey = efa_query_pkey,
 	.query_port = efa_query_port,
 	.query_qp = efa_query_qp,
-	.reg_user_mr = efa_reg_mr,
+	.reg_user_mr = efa_reg_user_mr,
 	.reg_user_mr_dmabuf = efa_reg_user_mr_dmabuf,
 
 	INIT_RDMA_OBJ_SIZE(ib_ah, efa_ah, ibah),
drivers/infiniband/hw/efa/efa_verbs.c +128 -157

742 lines not on this page · raw message

Signature · 1 line
2.40.1
maintainer
#14 lore raw

Re: [PATCH for-next 5/5] RDMA/efa: Align private func names to a single convention

Quoted 3 lines · Michael Margolin

Name functions that aren't exposed outside of a file with '_' prefix and make sure that all verbs ops' implementations are named according to the op name with an additional 'efa_' prefix.

That isn't the kernel convention, please don't use _ like this

Jason

author maintainer
#15 lore raw

Re: [PATCH for-next 5/5] RDMA/efa: Align private func names to a single convention

Quoted 11 lines · Jason Gunthorpe

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

Quoted 3 lines · Michael Margolin

Name functions that aren't exposed outside of a file with '_' prefix and make sure that all verbs ops' implementations are named according to the op name with an additional 'efa_' prefix.

That isn't the kernel convention, please don't use _ like this

Jason

AFAIK there is no single kernel convention for static functions naming and it also seems that the underscore prefix isn't rare in the subsystem.

Would you suggest a preferred style?

Michael

#16 lore raw

Re: [PATCH for-next 5/5] RDMA/efa: Align private func names to a single convention

Quoted 10 lines · Michael
Quoted 8 lines · Jason Gunthorpe

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

Quoted 3 lines · Michael Margolin

Name functions that aren't exposed outside of a file with '_' prefix and make sure that all verbs ops' implementations are named according to the op name with an additional 'efa_' prefix.

Quoted 6 lines
Quoted 3 lines

That isn't the kernel convention, please don't use _ like this

Jason

AFAIK there is no single kernel convention for static functions naming and it also seems that the underscore prefix isn't rare in the subsystem.

Underscore at the beginning of the function name is a common practice to mark that function as locked variant.

Thanks

maintainer
#17 lore raw

Re: [PATCH for-next 5/5] RDMA/efa: Align private func names to a single convention

Quoted 13 lines · Leon Romanovsky
Quoted 12 lines · Michael
Quoted 10 lines · Jason Gunthorpe

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

Quoted 3 lines · Michael Margolin

Name functions that aren't exposed outside of a file with '_' prefix and make sure that all verbs ops' implementations are named according to the op name with an additional 'efa_' prefix.

That isn't the kernel convention, please don't use _ like this

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Jason

AFAIK there is no single kernel convention for static functions naming and it also seems that the underscore prefix isn't rare in the subsystem.

Underscore at the beginning of the function name is a common practice to mark that function as locked variant.

Yeah, it is not typical to mark static functions at all.

You might exclude the symbol prefix (like efa_) but even that is not universal.

Jason

maintainer
#18 lore raw
In reply to Michael Margolin
Quoted 2 lines · Michael Margolin

RDMA/efa: Use offset_in_page() function
RDMA/efa: Remove duplicate aenq enable macro

I took these two into for-next. It seems the others have comments.

Thanks,
Jason