IB/hfi1: Fixes for rc or next if too late

PATCH Applied Dennis Dalessandro 3 participants 7 messages May 31, 2018 – Jun 4, 2018

Subsystems (3)

Stable branch InfiniBand HFI1 driver

lore

Share & tools

lore

Dennis Dalessandro

author

2018-05-31 18:29 UTC #0 lore raw

cover IB/hfi1: Fixes for rc or next if too late

Hi Doug and Jason,

We have two more late breaking fix up patches. The DMA_RTAIL fix is the more serious of the two. I realize we are at the tail end of 4.17 so I would not be against holding off till 4.18 for these, but if there is another rdma pull request we may want to tack these on.

---

Kaike Wan (1):
IB/hfi1: Ensure VL index is within bounds

Mike Marciniszyn (1):
IB/hfi1: Fix user context tail allocation for DMA_RTAIL

 drivers/infiniband/hw/hfi1/chip.c     |    8 ++++----
 drivers/infiniband/hw/hfi1/file_ops.c |    2 +-
 drivers/infiniband/hw/hfi1/init.c     |    9 ++++-----
 drivers/infiniband/hw/hfi1/sdma.c     |   12 +++---------
 4 files changed, 12 insertions(+), 19 deletions(-)

Dennis Dalessandro

author

2018-05-31 18:30 UTC #1 lore raw

PATCH 1/2 IB/hfi1: Fix user context tail allocation for DMA_RTAIL

From: Mike Marciniszyn <[email protected]>

The following code fails to allocate a buffer for the tail address that the hardware DMAs into when the user context DMA_RTAIL is set.

if (HFI1_CAP_KGET_MASK(rcd->flags, DMA_RTAIL)) {
	rcd->rcvhdrtail_kvaddr = dma_zalloc_coherent(
		&dd->pcidev->dev, PAGE_SIZE, &dma_hdrqtail,
                gfp_flags);
	if (!rcd->rcvhdrtail_kvaddr)
		goto bail_free;
	rcd->rcvhdrqtailaddr_dma = dma_hdrqtail;
}

So the rcvhdrtail_kvaddr would then be NULL.

The mmap logic fails to check for a NULL rcvhdrtail_kvaddr.

The fix is to test for both user and kernel DMA_TAIL options during the allocation as well as testing for a NULL rcvhdrtail_kvaddr during the mmap processing.

Additionally, all downstream testing of the capmask for DMA_RTAIL have been eliminated in favor of testing rcvhdrtail_kvaddr.

Cc <[email protected]> # 4.9.x

Reviewed-by Michael J. Ruhl <[email protected]>

Signed-off-by Mike Marciniszyn <[email protected]>

Signed-off-by Dennis Dalessandro <[email protected]>

---
 drivers/infiniband/hw/hfi1/chip.c     |    8 ++++----
 drivers/infiniband/hw/hfi1/file_ops.c |    2 +-
 drivers/infiniband/hw/hfi1/init.c     |    9 ++++-----
 3 files changed, 9 insertions(+), 10 deletions(-)

drivers/infiniband/hw/hfi1/chip.c +4 -4

index 9b1f97f..ccbdce2 100644
@@ -6841,7 +6841,7 @@ static void rxe_kernel_unfreeze(struct hfi1_devdata *dd)
 		}
 		rcvmask = HFI1_RCVCTRL_CTXT_ENB;
 		/* HFI1_RCVCTRL_TAILUPD_[ENB|DIS] needs to be set explicitly */
-		rcvmask |= HFI1_CAP_KGET_MASK(rcd->flags, DMA_RTAIL) ?
+		rcvmask |= rcd->rcvhdrtail_kvaddr ?
 			HFI1_RCVCTRL_TAILUPD_ENB : HFI1_RCVCTRL_TAILUPD_DIS;
 		hfi1_rcvctrl(dd, rcvmask, rcd);
 		hfi1_rcd_put(rcd);

@@ -8367,7 +8367,7 @@ static inline int check_packet_present(struct hfi1_ctxtdata *rcd)
 	u32 tail;
 	int present;
 
-	if (!HFI1_CAP_IS_KSET(DMA_RTAIL))
+	if (!rcd->rcvhdrtail_kvaddr)
 		present = (rcd->seq_cnt ==
 				rhf_rcv_seq(rhf_to_cpu(get_rhf_addr(rcd))));
 	else /* is RDMA rtail */

@@ -11843,7 +11843,7 @@ void hfi1_rcvctrl(struct hfi1_devdata *dd, unsigned int op,
 		/* reset the tail and hdr addresses, and sequence count */
 		write_kctxt_csr(dd, ctxt, RCV_HDR_ADDR,
 				rcd->rcvhdrq_dma);
-		if (HFI1_CAP_KGET_MASK(rcd->flags, DMA_RTAIL))
+		if (rcd->rcvhdrtail_kvaddr)
 			write_kctxt_csr(dd, ctxt, RCV_HDR_TAIL_ADDR,
 					rcd->rcvhdrqtailaddr_dma);
 		rcd->seq_cnt = 1;

@@ -11923,7 +11923,7 @@ void hfi1_rcvctrl(struct hfi1_devdata *dd, unsigned int op,
 		rcvctrl |= RCV_CTXT_CTRL_INTR_AVAIL_SMASK;
 	if (op & HFI1_RCVCTRL_INTRAVAIL_DIS)
 		rcvctrl &= ~RCV_CTXT_CTRL_INTR_AVAIL_SMASK;
-	if (op & HFI1_RCVCTRL_TAILUPD_ENB && rcd->rcvhdrqtailaddr_dma)
+	if ((op & HFI1_RCVCTRL_TAILUPD_ENB) && rcd->rcvhdrtail_kvaddr)
 		rcvctrl |= RCV_CTXT_CTRL_TAIL_UPD_SMASK;
 	if (op & HFI1_RCVCTRL_TAILUPD_DIS) {
 		/* See comment on RcvCtxtCtrl.TailUpd above */

drivers/infiniband/hw/hfi1/file_ops.c +1 -1

index c9d23c3..0fc4aa9 100644
@@ -505,7 +505,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
 			ret = -EINVAL;
 			goto done;
 		}
-		if (flags & VM_WRITE) {
+		if ((flags & VM_WRITE) || !uctxt->rcvhdrtail_kvaddr) {
 			ret = -EPERM;
 			goto done;
 		}

drivers/infiniband/hw/hfi1/init.c +4 -5

index 8265551..f2a0b03 100644
@@ -1838,7 +1838,6 @@ int hfi1_create_rcvhdrq(struct hfi1_devdata *dd, struct hfi1_ctxtdata *rcd)
 	u64 reg;
 
 	if (!rcd->rcvhdrq) {
-		dma_addr_t dma_hdrqtail;
 		gfp_t gfp_flags;
 
 		/*

@@ -1863,13 +1862,13 @@ int hfi1_create_rcvhdrq(struct hfi1_devdata *dd, struct hfi1_ctxtdata *rcd)
 			goto bail;
 		}
 
-		if (HFI1_CAP_KGET_MASK(rcd->flags, DMA_RTAIL)) {
+		if (HFI1_CAP_KGET_MASK(rcd->flags, DMA_RTAIL) ||
+		    HFI1_CAP_UGET_MASK(rcd->flags, DMA_RTAIL)) {
 			rcd->rcvhdrtail_kvaddr = dma_zalloc_coherent(
-				&dd->pcidev->dev, PAGE_SIZE, &dma_hdrqtail,
-				gfp_flags);
+				&dd->pcidev->dev, PAGE_SIZE,
+				&rcd->rcvhdrqtailaddr_dma, gfp_flags);
 			if (!rcd->rcvhdrtail_kvaddr)
 				goto bail_free;
-			rcd->rcvhdrqtailaddr_dma = dma_hdrqtail;
 		}
 
 		rcd->rcvhdrq_size = amt;

Doug Ledford

2018-05-31 18:47 UTC #2 lore raw

In reply to Dennis Dalessandro

Quoted 16 lines · Dennis Dalessandro

Hi Doug and Jason,
We have two more late breaking fix up patches. The DMA_RTAIL fix is the more serious of the two. I realize we are at the tail end of 4.17 so I would not be against holding off till 4.18 for these, but if there is another rdma pull request we may want to tack these on.
---
Kaike Wan (1):
IB/hfi1: Ensure VL index is within bounds
Mike Marciniszyn (1):
IB/hfi1: Fix user context tail allocation for DMA_RTAIL

Quoted 8 lines

 drivers/infiniband/hw/hfi1/chip.c     |    8 ++++----
 drivers/infiniband/hw/hfi1/file_ops.c |    2 +-
 drivers/infiniband/hw/hfi1/init.c     |    9 ++++-----
 drivers/infiniband/hw/hfi1/sdma.c     |   12 +++---------
 4 files changed, 12 insertions(+), 19 deletions(-)

--
-Denny

Hi Denny,

These two patches look fine in terms of the patches themselves. In terms of whether to put them in for-rc or for-next, what's the consequences of hitting each of these bugs?

Signature · 3 lines

Doug Ledford <[email protected]>
    GPG KeyID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

Dennis Dalessandro

author

2018-05-31 19:08 UTC #3 lore raw

Quoted 23 lines · Doug Ledford

Quoted 22 lines · Dennis Dalessandro
Hi Doug and Jason,
We have two more late breaking fix up patches. The DMA_RTAIL fix is the more serious of the two. I realize we are at the tail end of 4.17 so I would not be against holding off till 4.18 for these, but if there is another rdma pull request we may want to tack these on.
---
Kaike Wan (1):
IB/hfi1: Ensure VL index is within bounds
Mike Marciniszyn (1):
IB/hfi1: Fix user context tail allocation for DMA_RTAIL
  drivers/infiniband/hw/hfi1/chip.c     |    8 ++++----
  drivers/infiniband/hw/hfi1/file_ops.c |    2 +-
  drivers/infiniband/hw/hfi1/init.c     |    9 ++++-----
  drivers/infiniband/hw/hfi1/sdma.c     |   12 +++---------
  4 files changed, 12 insertions(+), 19 deletions(-)

Quoted 9 lines

Quoted 2 lines
--
-Denny
Hi Denny,
These two patches look fine in terms of the patches themselves. In terms of whether to put them in for-rc or for-next, what's the consequences of hitting each of these bugs?

The VL index, could be bad because it would jump beyond the end of the array. However, we won't actually hit that with the code the way it currently is because of the way we validate the VL in other areas of the code. This is more of a we better fix it before we do end up with a problem sort of thing.

In the other one, the DMA_RTAIL one, the driver ends up mmaping NULL and handing that user space. This only happens though if users muck with the CAP_MASK and enable the dma of the rtail. Which is not the default. Mike found this through code inspection I believe.

So they do fix serious flaws, but the likelihood of actually hitting them is very slim. Based on the stable tag on Mike's patch we have had this since 4.9.

-Denny

Jason Gunthorpe

2018-05-31 20:21 UTC #4 lore raw

Quoted 31 lines · Dennis Dalessandro

Quoted 23 lines · Doug Ledford
Quoted 15 lines · Dennis Dalessandro
Hi Doug and Jason,
We have two more late breaking fix up patches. The DMA_RTAIL fix is the more serious of the two. I realize we are at the tail end of 4.17 so I would not be against holding off till 4.18 for these, but if there is another rdma pull request we may want to tack these on.
Kaike Wan (1):
IB/hfi1: Ensure VL index is within bounds
Mike Marciniszyn (1):
IB/hfi1: Fix user context tail allocation for DMA_RTAIL
Quoted 6 lines
 drivers/infiniband/hw/hfi1/chip.c     |    8 ++++----
 drivers/infiniband/hw/hfi1/file_ops.c |    2 +-
 drivers/infiniband/hw/hfi1/init.c     |    9 ++++-----
 drivers/infiniband/hw/hfi1/sdma.c     |   12 +++---------
 4 files changed, 12 insertions(+), 19 deletions(-)
Quoted 6 lines
Hi Denny,
These two patches look fine in terms of the patches themselves. In terms of whether to put them in for-rc or for-next, what's the consequences of hitting each of these bugs?

Quoted 5 lines

The VL index, could be bad because it would jump beyond the end of the array. However, we won't actually hit that with the code the way it currently is because of the way we validate the VL in other areas of the code. This is more of a we better fix it before we do end up with a problem sort of thing.

Theoretical future bugs are not rc or stable material

Quoted 8 lines

In the other one, the DMA_RTAIL one, the driver ends up mmaping NULL and handing that user space. This only happens though if users muck with the CAP_MASK and enable the dma of the rtail. Which is not the default. Mike found this through code inspection I believe.
So they do fix serious flaws, but the likelihood of actually hitting them is very slim. Based on the stable tag on Mike's patch we have had this since 4.9.

I think it is too late for more -rc stuff..

The last -rc (assuming rc7 is the end) pull request needs to go tomorrow morning and we like it to have -rc stuff sit in -next for at least a day before sending to Linus :\

Jason

Dennis Dalessandro

author

2018-05-31 20:29 UTC #5 lore raw

Quoted 32 lines · Jason Gunthorpe

Quoted 31 lines · Dennis Dalessandro
Quoted 29 lines · Doug Ledford
Quoted 15 lines · Dennis Dalessandro
Hi Doug and Jason,
We have two more late breaking fix up patches. The DMA_RTAIL fix is the more serious of the two. I realize we are at the tail end of 4.17 so I would not be against holding off till 4.18 for these, but if there is another rdma pull request we may want to tack these on.
Kaike Wan (1):
IB/hfi1: Ensure VL index is within bounds
Mike Marciniszyn (1):
IB/hfi1: Fix user context tail allocation for DMA_RTAIL
Quoted 6 lines
  drivers/infiniband/hw/hfi1/chip.c     |    8 ++++----
  drivers/infiniband/hw/hfi1/file_ops.c |    2 +-
  drivers/infiniband/hw/hfi1/init.c     |    9 ++++-----
  drivers/infiniband/hw/hfi1/sdma.c     |   12 +++---------
  4 files changed, 12 insertions(+), 19 deletions(-)
Hi Denny,
These two patches look fine in terms of the patches themselves. In terms of whether to put them in for-rc or for-next, what's the consequences of hitting each of these bugs?

Quoted 7 lines

Quoted 5 lines
The VL index, could be bad because it would jump beyond the end of the array. However, we won't actually hit that with the code the way it currently is because of the way we validate the VL in other areas of the code. This is more of a we better fix it before we do end up with a problem sort of thing.
Theoretical future bugs are not rc or stable material

The VL index one is not marked as stable, however the other one is. That is because it is a real bug, as Mike said to me earlier, we provide the tools to play, the concern is if someone does. So I think it meets stable bar, but not necessarily this late of an -rc.

Quoted 9 lines

Quoted 8 lines
In the other one, the DMA_RTAIL one, the driver ends up mmaping NULL and handing that user space. This only happens though if users muck with the CAP_MASK and enable the dma of the rtail. Which is not the default. Mike found this through code inspection I believe.
So they do fix serious flaws, but the likelihood of actually hitting them is very slim. Based on the stable tag on Mike's patch we have had this since 4.9.

Quoted 5 lines

I think it is too late for more -rc stuff..
The last -rc (assuming rc7 is the end) pull request needs to go tomorrow morning and we like it to have -rc stuff sit in -next for at least a day before sending to Linus :\

Agree, and fine with me.

-Denny

Jason Gunthorpe

Applied

2018-06-04 19:34 UTC #6 lore raw

In reply to Dennis Dalessandro

Quoted 8 lines · Dennis Dalessandro

Hi Doug and Jason,
We have two more late breaking fix up patches. The DMA_RTAIL fix is the more serious of the two. I realize we are at the tail end of 4.17 so I would not be against holding off till 4.18 for these, but if there is another rdma pull request we may want to tack these on.

Quoted 5 lines

Kaike Wan (1):
IB/hfi1: Ensure VL index is within bounds
Mike Marciniszyn (1):
IB/hfi1: Fix user context tail allocation for DMA_RTAIL

applied to for-next since the merge window is open now

Thanks,
Jason