Skip to contentSkip to conversation

IB/hfi1: PCI bug due to pci core changes

PATCH Applied Dennis Dalessandro 4 participants 9 messages Aug 31, 2018 – Sep 12, 2018

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover IB/hfi1: PCI bug due to pci core changes

Hi Bjorn, Doug and Jason,

As discussed on the pci list [1] our driver was broken with the following three changes in 4.19 merge window:

c6a44ba950d1 ("PCI: Rename pci_try_reset_bus() to pci_reset_bus()") 409888e0966e ("IB/hfi1: Use pci_try_reset_bus() for initiating PCI Secondary Bus Reset") 811c5cb37df4 ("PCI: Unify try slot and bus reset API")

Ideally I'd like to see those patches reverted but we could also go with something like the following series. I know there is a desire to have a more clean API but since this is already the rc phase I think that is best delayed. Clearly the first attempt at an API clean up did not work out so well.

I'm not sure how you maintainers want to handle this since the fix straddles both subsystems so I'm sending it to both linux-rdma and linux-pci.

At the request of discussion on the linux-pci list I have also filed a bug [2].

[1] https://marc.info/?l=linux-pci&m=153539692917785&w=2
[2] https://bugzilla.kernel.org/show_bug.cgi?id=200985

Thanks

---

Dennis Dalessandro (2):
PCI: Fix faulty logic in pci_reset_bus()
IB/hfi1,PCI: Allow bus reset while probing

 drivers/infiniband/hw/hfi1/pcie.c |   11 ++++-------
 drivers/pci/pci.c                 |    3 ++-
 include/linux/pci.h               |    3 +++
 3 files changed, 9 insertions(+), 8 deletions(-)

--
-Denny

#1 lore raw

PATCH 1/2 PCI: Fix faulty logic in pci_reset_bus()

The pci_rest_bus() function calls into pci_probe_reset_slot() to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account for pci_probe_reset_slot() being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.

Cc Sinan Kaya <[email protected]>
Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")
---
 drivers/pci/pci.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/pci/pci.c +1 -1
index 29ff961..30b2603 100644
@@ -5200,7 +5200,7 @@ static int __pci_reset_bus(struct pci_bus *bus)
  */
 int pci_reset_bus(struct pci_dev *pdev)
 {
-	return pci_probe_reset_slot(pdev->slot) ?
+	return (!pci_probe_reset_slot(pdev->slot)) ?
 	    __pci_reset_slot(pdev->slot) : __pci_reset_bus(pdev->bus);
 }
 EXPORT_SYMBOL_GPL(pci_reset_bus);
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH for-rc 1/2] PCI: Fix faulty logic in pci_reset_bus()

Quoted 10 lines · Dennis Dalessandro

The pci_rest_bus() function calls into pci_probe_reset_slot() to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account for pci_probe_reset_slot() being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.

Cc Sinan Kaya<[email protected]>
Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")

Nit. Small typo on the first sentence (pci_rest_bus()).

#3 lore raw

Re: [PATCH for-rc 1/2] PCI: Fix faulty logic in pci_reset_bus()

Quoted 7 lines · Sinan Kaya
Quoted 6 lines · Dennis Dalessandro

The pci_rest_bus() function calls into pci_probe_reset_slot() to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account for pci_probe_reset_slot() being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.

Quoted 8 lines
Quoted 4 lines
Cc Sinan Kaya<[email protected]>
Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")

Nit. Small typo on the first sentence (pci_rest_bus()).

Bjorn,

Are you OK to apply this series through the RDMA tree (for rc3), or do you want to take it through PCI?

https://patchwork.kernel.org/patch/10584277/

Thanks,
Jason

#4 lore raw

Re: [PATCH for-rc 1/2] PCI: Fix faulty logic in pci_reset_bus()

Quoted 15 lines · Jason Gunthorpe
Quoted 14 lines · Sinan Kaya
Quoted 10 lines · Dennis Dalessandro

The pci_rest_bus() function calls into pci_probe_reset_slot() to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account for pci_probe_reset_slot() being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.

Cc Sinan Kaya<[email protected]>
Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")

Nit. Small typo on the first sentence (pci_rest_bus()).

Quoted 8 lines
Quoted 1 line

Bjorn,

Are you OK to apply this series through the RDMA tree (for rc3), or do you want to take it through PCI?

https://patchwork.kernel.org/patch/10584277/

Please don't apply the entire series yet. First patch is good to go.

Second one is a hack. We are trying to find a better solution for the second patch.

https://bugzilla.kernel.org/show_bug.cgi?id=200985

Quoted 4 lines

Thanks,
Jason

#5 lore raw

Re: [PATCH for-rc 1/2] PCI: Fix faulty logic in pci_reset_bus()

Quoted 23 lines · Sinan Kaya
Quoted 22 lines · Jason Gunthorpe
Quoted 7 lines · Sinan Kaya
Quoted 6 lines · Dennis Dalessandro

The pci_rest_bus() function calls into pci_probe_reset_slot() to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account for pci_probe_reset_slot() being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.

Quoted 8 lines
Quoted 4 lines
Cc Sinan Kaya<[email protected]>
Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")

Nit. Small typo on the first sentence (pci_rest_bus()).

Bjorn,

Are you OK to apply this series through the RDMA tree (for rc3), or do you want to take it through PCI?

Quoted 6 lines
Quoted 1 line

https://patchwork.kernel.org/patch/10584277/

Please don't apply the entire series yet. First patch is good to go.

Second one is a hack. We are trying to find a better solution for the second patch.

Don't expect me to follow bugzilla too :|

I'll drop this series off patchworks then, resend when you have something..

Jason

#6 lore raw

Re: [PATCH for-rc 1/2] PCI: Fix faulty logic in pci_reset_bus()

Quoted 26 lines · Jason Gunthorpe
Quoted 25 lines · Sinan Kaya
Quoted 23 lines · Jason Gunthorpe
Quoted 7 lines · Sinan Kaya
Quoted 6 lines · Dennis Dalessandro

The pci_rest_bus() function calls into pci_probe_reset_slot() to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account for pci_probe_reset_slot() being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.

Quoted 8 lines
Quoted 4 lines
Cc Sinan Kaya<[email protected]>
Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")

Nit. Small typo on the first sentence (pci_rest_bus()).

Bjorn,

Are you OK to apply this series through the RDMA tree (for rc3), or do you want to take it through PCI?

https://patchwork.kernel.org/patch/10584277/

Quoted 9 lines
Quoted 4 lines

Please don't apply the entire series yet. First patch is good to go.

Second one is a hack. We are trying to find a better solution for the second patch.

Don't expect me to follow bugzilla too :|

I'll drop this series off patchworks then, resend when you have something..

Sure, I'm trying to contain the change to PCI tree only. Hopefully, it can go
through Bjorn's tree. If we fail, we are back to this change. I'll ping you in that case.

Quoted 2 lines

Jason

#7 lore raw

PATCH 2/2 IB/hfi1,PCI: Allow bus reset while probing

In reply to Dennis Dalessandro

Calling into the new API to reset the secondary bus results in a deadlock. This occurs because the device/bus is already locked at probe time. Reverting back to the old behavior while the API is improved.

Fixes c6a44ba950d1 ("PCI: Rename pci_try_reset_bus() to pci_reset_bus()")
Fixes 409888e0966e ("IB/hfi1: Use pci_try_reset_bus() for initiating PCI Secondary Bus Reset")
Cc Sinan Kaya <[email protected]>
---
 drivers/infiniband/hw/hfi1/pcie.c |   11 ++++-------
 drivers/pci/pci.c                 |    1 +
 include/linux/pci.h               |    3 +++
 3 files changed, 8 insertions(+), 7 deletions(-)
drivers/infiniband/hw/hfi1/pcie.c +4 -7
index eec8375..6c967dd 100644
@@ -893,14 +893,11 @@ static int trigger_sbr(struct hfi1_devdata *dd)
 		}
 
 	/*
-	 * A secondary bus reset (SBR) issues a hot reset to our device.
-	 * The following routine does a 1s wait after the reset is dropped
-	 * per PCI Trhfa (recovery time).  PCIe 3.0 section 6.6.1 -
-	 * Conventional Reset, paragraph 3, line 35 also says that a 1s
-	 * delay after a reset is required.  Per spec requirements,
-	 * the link is either working or not after that point.
+	 * This is an end around to do an SBR during probe time. A new API needs
+	 * to be implemented to have cleaner interface but this fixes the
+	 * current brokenness
 	 */
-	return pci_reset_bus(dev);
+	return pci_bridge_secondary_bus_reset(dev->bus->self);
 }
 
 /*
drivers/pci/pci.c +1 -0
index 30b2603..1835f3a 100644
@@ -4547,6 +4547,7 @@ int pci_bridge_secondary_bus_reset(struct pci_dev *dev)
 
 	return pci_dev_wait(dev, "bus reset", PCIE_RESET_READY_POLL_MS);
 }
+EXPORT_SYMBOL_GPL(pci_bridge_secondary_bus_reset);
 
 static int pci_parent_bus_reset(struct pci_dev *dev, int probe)
 {
include/linux/pci.h +3 -0
index e72ca8d..6925828 100644
@@ -1235,6 +1235,9 @@ void pci_bus_add_resource(struct pci_bus *bus, struct resource *res,
 int devm_request_pci_bus_resources(struct device *dev,
 				   struct list_head *resources);
 
+/* Temporary until new and working PCI SBR API in place */
+int pci_bridge_secondary_bus_reset(struct pci_dev *dev);
+
 #define pci_bus_for_each_resource(bus, res, i)				\
 	for (i = 0;							\
 	    (res = pci_bus_resource_n(bus, i)) || i < PCI_BRIDGE_RESOURCE_NUM; \
#8 lore raw
In reply to Dennis Dalessandro
Quoted 32 lines · Dennis Dalessandro

Hi Bjorn, Doug and Jason,

As discussed on the pci list [1] our driver was broken with the following three changes in 4.19 merge window:

c6a44ba950d1 ("PCI: Rename pci_try_reset_bus() to pci_reset_bus()") 409888e0966e ("IB/hfi1: Use pci_try_reset_bus() for initiating PCI Secondary Bus Reset") 811c5cb37df4 ("PCI: Unify try slot and bus reset API")

Ideally I'd like to see those patches reverted but we could also go with something like the following series. I know there is a desire to have a more clean API but since this is already the rc phase I think that is best delayed. Clearly the first attempt at an API clean up did not work out so well.

I'm not sure how you maintainers want to handle this since the fix straddles both subsystems so I'm sending it to both linux-rdma and linux-pci.

At the request of discussion on the linux-pci list I have also filed a bug [2].

[1] https://marc.info/?l=linux-pci&m=153539692917785&w=2
[2] https://bugzilla.kernel.org/show_bug.cgi?id=200985

Thanks

---

Dennis Dalessandro (2):
PCI: Fix faulty logic in pci_reset_bus()
IB/hfi1,PCI: Allow bus reset while probing

Quoted 4 lines
 drivers/infiniband/hw/hfi1/pcie.c |   11 ++++-------
 drivers/pci/pci.c                 |    3 ++-
 include/linux/pci.h               |    3 +++
 3 files changed, 9 insertions(+), 8 deletions(-)

I applied both of these to for-linus for v4.19, thanks!