Skip to contentSkip to conversation

RDMA/mlx5: Fix user-triggerable overflows in QP creation

PATCH In review Edward Srouji 2 participants 7 messages Jun 11 – Jun 15

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover RDMA/mlx5: Fix user-triggerable overflows in QP creation

This short series fixes two undefined behavior bugs in user QP creation path.

---
Maher Sanalla (2):
RDMA/mlx5: Fix undefined shift of user RQ WQE size RDMA/mlx5: Fix integer overflow of user QP buffer size

 drivers/infiniband/hw/mlx5/qp.c | 54 ++++++++++++++++++++++++++++++++---------
 1 file changed, 42 insertions(+), 12 deletions(-)

---
base-commit: 20ff9350862468af21b46cae2c22d17d6ec637f9
change-id: 20260611-maher-sec-fixes-4cd89b9fe4bd

Best regards,

Signature · 1 line
Edward Srouji <[email protected]>
#1 lore raw

PATCH 1/2 RDMA/mlx5: Fix undefined shift of user RQ WQE size

From: Maher Sanalla <[email protected]>

set_rq_size() computes the RQ WQE size as "1 << rq_wqe_shift" based on the user-provided rq_wqe_shift, which is only checked to be greater than 32, so shifts of 32 are still accepted. A shift of 31 also overflows a signed integer, leading to undefined behavior.

Use check_shl_overflow() to compute the RQ WQE size and reject any invalid values.

Fixes e126ba97dba9 ("mlx5: Add driver for Mellanox Connect-IB adapters")
---
 drivers/infiniband/hw/mlx5/qp.c | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)
drivers/infiniband/hw/mlx5/qp.c +4 -7
index e8d34d54b43527e0595ec9e2fb93dc7e9bedba92..7674290d0afaf466a6b98cbed86d247ee550bd8d 100644
@@ -454,16 +454,13 @@ static int set_rq_size(struct mlx5_ib_dev *dev, struct ib_qp_cap *cap,
 
 		if (ucmd) {
 			qp->rq.wqe_cnt = ucmd->rq_wqe_count;
-			if (ucmd->rq_wqe_shift > BITS_PER_BYTE * sizeof(ucmd->rq_wqe_shift))
-				return -EINVAL;
 			qp->rq.wqe_shift = ucmd->rq_wqe_shift;
-			if ((1 << qp->rq.wqe_shift) /
-				    sizeof(struct mlx5_wqe_data_seg) <
-			    wq_sig)
+			if (check_shl_overflow(1, qp->rq.wqe_shift, &wqe_size))
+				return -EINVAL;
+			if (wqe_size / sizeof(struct mlx5_wqe_data_seg) < wq_sig)
 				return -EINVAL;
 			qp->rq.max_gs =
-				(1 << qp->rq.wqe_shift) /
-					sizeof(struct mlx5_wqe_data_seg) -
+				wqe_size / sizeof(struct mlx5_wqe_data_seg) -
 				wq_sig;
 			qp->rq.max_post = qp->rq.wqe_cnt;
 		} else {
Signature · 1 line
2.49.0
maintainer
#2 lore raw

Re: [PATCH rdma-next 1/2] RDMA/mlx5: Fix undefined shift of user RQ WQE size

Quoted 10 lines · Edward Srouji

From: Maher Sanalla <[email protected]>

set_rq_size() computes the RQ WQE size as "1 << rq_wqe_shift" based on the user-provided rq_wqe_shift, which is only checked to be greater than 32, so shifts of 32 are still accepted. A shift of 31 also overflows a signed integer, leading to undefined behavior.

Use check_shl_overflow() to compute the RQ WQE size and reject any invalid values.

Quoted 6 lines
Fixes e126ba97dba9 ("mlx5: Add driver for Mellanox Connect-IB adapters")
---
 drivers/infiniband/hw/mlx5/qp.c | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

Applied just this one, please resend the other one

Thanks,
Jason

#3 lore raw

PATCH 2/2 RDMA/mlx5: Fix integer overflow of user QP buffer size

In reply to Edward Srouji

From: Maher Sanalla <[email protected]>

set_user_buf_size() calculates the QP buffer size by left‑shifting the user‑provided rq.wqe_cnt and rq.wqe_shift as signed integers. A large rq.wqe_cnt can trigger a signed integer overflow, which is undefined behavior and may yield a small or even negative buf_size. This can lead ib_umem_get() to map a buffer smaller than what the hardware will write.

Replace the existing shifts and additions with check_shl_overflow() and check_add_overflow(), and reject invalid user inputs.

Apply the same checks to the calculation used for qp->sq.offset in _create_user_qp(). Even though set_user_buf_size() validates this again, the guard protects us against future changes in the internal implementation.

Fixes e126ba97dba9 ("mlx5: Add driver for Mellanox Connect-IB adapters")
---
 drivers/infiniband/hw/mlx5/qp.c | 43 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 38 insertions(+), 5 deletions(-)
drivers/infiniband/hw/mlx5/qp.c +38 -5
index 7674290d0afaf466a6b98cbed86d247ee550bd8d..6ecdbda2b471f6c102bceba5d02eb12af8d8e1b1 100644
@@ -640,6 +640,8 @@ static int set_user_buf_size(struct mlx5_ib_dev *dev,
 			    struct ib_qp_init_attr *attr)
 {
 	int desc_sz = 1 << qp->sq.wqe_shift;
+	int rq_buf_size;
+	int sq_buf_size;
 
 	if (desc_sz > MLX5_CAP_GEN(dev->mdev, max_wqe_sz_sq)) {
 		mlx5_ib_warn(dev, "desc_sz %d, max_sq_desc_sz %d\n",
@@ -664,11 +666,36 @@ static int set_user_buf_size(struct mlx5_ib_dev *dev,
 
 	if (attr->qp_type == IB_QPT_RAW_PACKET ||
 	    qp->flags & IB_QP_CREATE_SOURCE_QPN) {
-		base->ubuffer.buf_size = qp->rq.wqe_cnt << qp->rq.wqe_shift;
-		qp->raw_packet_qp.sq.ubuffer.buf_size = qp->sq.wqe_cnt << 6;
+		if (check_shl_overflow(qp->rq.wqe_cnt, qp->rq.wqe_shift,
+				       &base->ubuffer.buf_size)) {
+			mlx5_ib_warn(dev, "rq buf size overflow: wqe_cnt %d wqe_shift %d\n",
+				     qp->rq.wqe_cnt, qp->rq.wqe_shift);
+			return -EINVAL;
+		}
+		if (check_shl_overflow(qp->sq.wqe_cnt, 6,
+				       &qp->raw_packet_qp.sq.ubuffer.buf_size)) {
+			mlx5_ib_warn(dev, "sq buf size overflow: wqe_cnt %d\n",
+				     qp->sq.wqe_cnt);
+			return -EINVAL;
+		}
 	} else {
-		base->ubuffer.buf_size = (qp->rq.wqe_cnt << qp->rq.wqe_shift) +
-					 (qp->sq.wqe_cnt << 6);
+		if (check_shl_overflow(qp->rq.wqe_cnt, qp->rq.wqe_shift,
+				       &rq_buf_size)) {
+			mlx5_ib_warn(dev, "rq buf size overflow: wqe_cnt %d wqe_shift %d\n",
+				     qp->rq.wqe_cnt, qp->rq.wqe_shift);
+			return -EINVAL;
+		}
+		if (check_shl_overflow(qp->sq.wqe_cnt, 6, &sq_buf_size)) {
+			mlx5_ib_warn(dev, "sq buf size overflow: wqe_cnt %d\n",
+				     qp->sq.wqe_cnt);
+			return -EINVAL;
+		}
+		if (check_add_overflow(rq_buf_size, sq_buf_size,
+				       &base->ubuffer.buf_size)) {
+			mlx5_ib_warn(dev, "qp buf size overflow: rq %d sq %d\n",
+				     rq_buf_size, sq_buf_size);
+			return -EINVAL;
+		}
 	}
 
 	return 0;
@@ -997,7 +1024,13 @@ static int _create_user_qp(struct mlx5_ib_dev *dev, struct ib_pd *pd,
 
 	qp->rq.offset = 0;
 	qp->sq.wqe_shift = ilog2(MLX5_SEND_WQE_BB);
-	qp->sq.offset = qp->rq.wqe_cnt << qp->rq.wqe_shift;
+	if (check_shl_overflow(qp->rq.wqe_cnt, qp->rq.wqe_shift,
+			       &qp->sq.offset)) {
+		mlx5_ib_warn(dev, "sq offset overflow: wqe_cnt %d wqe_shift %d\n",
+			     qp->rq.wqe_cnt, qp->rq.wqe_shift);
+		err = -EINVAL;
+		goto err_bfreg;
+	}
 
 	err = set_user_buf_size(dev, qp, ucmd, base, attr);
 	if (err)
Signature · 1 line
2.49.0
maintainer
#4 lore raw

Re: [PATCH rdma-next 2/2] RDMA/mlx5: Fix integer overflow of user QP buffer size

drivers/infiniband/hw/mlx5/qp.c static int set_user_buf_size(struct mlx5_ib_dev *dev, · line 673

Quoted 11 lines · Edward Srouji
drivers/infiniband/hw/mlx5/qp.c +5 -2
@@ -664,11 +666,36 @@ static int set_user_buf_size(struct mlx5_ib_dev *dev,
 
 	if (attr->qp_type == IB_QPT_RAW_PACKET ||
 	    qp->flags & IB_QP_CREATE_SOURCE_QPN) {
-		base->ubuffer.buf_size = qp->rq.wqe_cnt << qp->rq.wqe_shift;
-		qp->raw_packet_qp.sq.ubuffer.buf_size = qp->sq.wqe_cnt << 6;
+		if (check_shl_overflow(qp->rq.wqe_cnt, qp->rq.wqe_shift,
+				       &base->ubuffer.buf_size)) {
+			mlx5_ib_warn(dev, "rq buf size overflow: wqe_cnt %d wqe_shift %d\n",
+				     qp->rq.wqe_cnt, qp->rq.wqe_shift);
+			return -EINVAL;

No prints triggerable by uapi.

Jason

#5 lore raw

Re: [PATCH rdma-next 2/2] RDMA/mlx5: Fix integer overflow of user QP buffer size

Quoted 15 lines · Jason Gunthorpe

drivers/infiniband/hw/mlx5/qp.c static int set_user_buf_size(struct mlx5_ib_dev *dev, · line 673

Quoted 11 lines · Edward Srouji
drivers/infiniband/hw/mlx5/qp.c +5 -2
@@ -664,11 +666,36 @@ static int set_user_buf_size(struct mlx5_ib_dev *dev,
 
  	if (attr->qp_type == IB_QPT_RAW_PACKET ||
  	    qp->flags & IB_QP_CREATE_SOURCE_QPN) {
-		base->ubuffer.buf_size = qp->rq.wqe_cnt << qp->rq.wqe_shift;
-		qp->raw_packet_qp.sq.ubuffer.buf_size = qp->sq.wqe_cnt << 6;
+		if (check_shl_overflow(qp->rq.wqe_cnt, qp->rq.wqe_shift,
+				       &base->ubuffer.buf_size)) {
+			mlx5_ib_warn(dev, "rq buf size overflow: wqe_cnt %d wqe_shift %d\n",
+				     qp->rq.wqe_cnt, qp->rq.wqe_shift);
+			return -EINVAL;

No prints triggerable by uapi.

Right, will drop them.
Note that set_user_buf_size() already has a pre-existing mlx5_ib_warn() prints, which is equally uapi-triggerable. Should we clean that up in a separate patch? Should we drop such prints entirely? or convert them to mlx5_ib_dbg()?

Quoted 1 line

Jason

maintainer
#6 lore raw

Re: [PATCH rdma-next 2/2] RDMA/mlx5: Fix integer overflow of user QP buffer size

Quoted 17 lines · Edward Srouji
Quoted 14 lines · Jason Gunthorpe

drivers/infiniband/hw/mlx5/qp.c static int set_user_buf_size(struct mlx5_ib_dev *dev, · line 672

Quoted 10 lines · Edward Srouji
drivers/infiniband/hw/mlx5/qp.c +5 -2
@@ -664,11 +666,36 @@ static int set_user_buf_size(struct mlx5_ib_dev *dev,
  	if (attr->qp_type == IB_QPT_RAW_PACKET ||
  	    qp->flags & IB_QP_CREATE_SOURCE_QPN) {
-		base->ubuffer.buf_size = qp->rq.wqe_cnt << qp->rq.wqe_shift;
-		qp->raw_packet_qp.sq.ubuffer.buf_size = qp->sq.wqe_cnt << 6;
+		if (check_shl_overflow(qp->rq.wqe_cnt, qp->rq.wqe_shift,
+				       &base->ubuffer.buf_size)) {
+			mlx5_ib_warn(dev, "rq buf size overflow: wqe_cnt %d wqe_shift %d\n",
+				     qp->rq.wqe_cnt, qp->rq.wqe_shift);
+			return -EINVAL;

No prints triggerable by uapi.

Quoted 5 lines

Right, will drop them.
Note that set_user_buf_size() already has a pre-existing mlx5_ib_warn() prints, which is equally uapi-triggerable.
Should we clean that up in a separate patch? Should we drop such prints entirely? or convert them to mlx5_ib_dbg()?

Yes clean them up, up to you if you want dbg versions

Jason