IB/hfi1: PCI bug due to pci core changes
Subsystems (3)
cover IB/hfi1: PCI bug due to pci core changes
Hi Bjorn, Doug and Jason,
As discussed on the pci list [1] our driver was broken with the following three changes in 4.19 merge window:
c6a44ba950d1 ("PCI: Rename pci_try_reset_bus() to pci_reset_bus()") 409888e0966e ("IB/hfi1: Use pci_try_reset_bus() for initiating PCI Secondary Bus Reset") 811c5cb37df4 ("PCI: Unify try slot and bus reset API")
Ideally I'd like to see those patches reverted but we could also go with something like the following series. I know there is a desire to have a more clean API but since this is already the rc phase I think that is best delayed. Clearly the first attempt at an API clean up did not work out so well.
I'm not sure how you maintainers want to handle this since the fix straddles both subsystems so I'm sending it to both linux-rdma and linux-pci.
At the request of discussion on the linux-pci list I have also filed a bug [2].
[1] https://marc.info/?l=linux-pci&m=153539692917785&w=2
[2] https://bugzilla.kernel.org/show_bug.cgi?id=200985
Thanks
---
Dennis Dalessandro (2):
PCI: Fix faulty logic in pci_reset_bus()
IB/hfi1,PCI: Allow bus reset while probing
drivers/infiniband/hw/hfi1/pcie.c | 11 ++++------- drivers/pci/pci.c | 3 ++- include/linux/pci.h | 3 +++ 3 files changed, 9 insertions(+), 8 deletions(-)
--
-Denny
PATCH 1/2 PCI: Fix faulty logic in pci_reset_bus()
The pci_rest_bus() function calls into pci_probe_reset_slot() to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account for pci_probe_reset_slot() being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.
--- drivers/pci/pci.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/pci/pci.c +1 -1
@@ -5200,7 +5200,7 @@ static int __pci_reset_bus(struct pci_bus *bus)*/ int pci_reset_bus (struct pci_dev *pdev) { -return pci_probe_reset_slot (pdev->slot) ? +return (!pci_probe_reset_slot (pdev->slot)) ?__pci_reset_slot (pdev->slot) :__pci_reset_bus (pdev->bus); }EXPORT_SYMBOL_GPL (pci_reset_bus);
Re: [PATCH for-rc 1/2] PCI: Fix faulty logic in pci_reset_bus()
Quoted 10 lines · Dennis Dalessandro
The
pci_rest_bus()function calls intopci_probe_reset_slot()to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account forpci_probe_reset_slot()being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.Cc Sinan Kaya<[email protected]>Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")Reviewed-by Michael J. Ruhl<[email protected]>Signed-off-by Dennis Dalessandro<[email protected]>
Nit. Small typo on the first sentence (pci_rest_bus()).
Re: [PATCH for-rc 1/2] PCI: Fix faulty logic in pci_reset_bus()
Quoted 7 lines · Sinan Kaya
Quoted 6 lines · Dennis Dalessandro
The
pci_rest_bus()function calls intopci_probe_reset_slot()to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account forpci_probe_reset_slot()being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.
Quoted 8 lines
Quoted 4 lines
Cc Sinan Kaya<[email protected]>Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")Reviewed-by Michael J. Ruhl<[email protected]>Signed-off-by Dennis Dalessandro<[email protected]>Nit. Small typo on the first sentence (
pci_rest_bus()).Reviewed-by Sinan Kaya <[email protected]>
Bjorn,
Are you OK to apply this series through the RDMA tree (for rc3), or do you want to take it through PCI?
https://patchwork.kernel.org/patch/10584277/
Thanks,
Jason
Re: [PATCH for-rc 1/2] PCI: Fix faulty logic in pci_reset_bus()
Quoted 15 lines · Jason Gunthorpe
Quoted 14 lines · Sinan Kaya
Quoted 10 lines · Dennis Dalessandro
The
pci_rest_bus()function calls intopci_probe_reset_slot()to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account forpci_probe_reset_slot()being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.Cc Sinan Kaya<[email protected]>Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")Reviewed-by Michael J. Ruhl<[email protected]>Signed-off-by Dennis Dalessandro<[email protected]>Nit. Small typo on the first sentence (
pci_rest_bus()).
Quoted 8 lines
Quoted 1 line
Reviewed-by Sinan Kaya <[email protected]>Bjorn,
Are you OK to apply this series through the RDMA tree (for rc3), or do you want to take it through PCI?
Please don't apply the entire series yet. First patch is good to go.
Second one is a hack. We are trying to find a better solution for the second patch.
https://bugzilla.kernel.org/show_bug.cgi?id=200985
Quoted 4 lines
Thanks,
Jason
Re: [PATCH for-rc 1/2] PCI: Fix faulty logic in pci_reset_bus()
Quoted 23 lines · Sinan Kaya
Quoted 22 lines · Jason Gunthorpe
Quoted 7 lines · Sinan Kaya
Quoted 6 lines · Dennis Dalessandro
The
pci_rest_bus()function calls intopci_probe_reset_slot()to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account forpci_probe_reset_slot()being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.Quoted 8 lines
Quoted 4 lines
Cc Sinan Kaya<[email protected]>Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")Reviewed-by Michael J. Ruhl<[email protected]>Signed-off-by Dennis Dalessandro<[email protected]>Nit. Small typo on the first sentence (
pci_rest_bus()).Reviewed-by Sinan Kaya <[email protected]>Bjorn,
Are you OK to apply this series through the RDMA tree (for rc3), or do you want to take it through PCI?
Quoted 6 lines
Quoted 1 line
Please don't apply the entire series yet. First patch is good to go.
Second one is a hack. We are trying to find a better solution for the second patch.
Don't expect me to follow bugzilla too :|
I'll drop this series off patchworks then, resend when you have something..
Jason
Re: [PATCH for-rc 1/2] PCI: Fix faulty logic in pci_reset_bus()
Quoted 26 lines · Jason Gunthorpe
Quoted 25 lines · Sinan Kaya
Quoted 23 lines · Jason Gunthorpe
Quoted 7 lines · Sinan Kaya
Quoted 6 lines · Dennis Dalessandro
The
pci_rest_bus()function calls intopci_probe_reset_slot()to determine whether to call the slot or bus reset. The check has faulty logic in that it does not account forpci_probe_reset_slot()being able to return an errno. Fix by only calling the slot reset when the function returns 0. Treat < 1 and > 1 the same.Quoted 8 lines
Quoted 4 lines
Cc Sinan Kaya<[email protected]>Fixes 811c5cb37df4 ("PCI: Unify try slot and bus reset API")Reviewed-by Michael J. Ruhl<[email protected]>Signed-off-by Dennis Dalessandro<[email protected]>Nit. Small typo on the first sentence (
pci_rest_bus()).Reviewed-by Sinan Kaya <[email protected]>Bjorn,
Are you OK to apply this series through the RDMA tree (for rc3), or do you want to take it through PCI?
Quoted 9 lines
Quoted 4 lines
Please don't apply the entire series yet. First patch is good to go.
Second one is a hack. We are trying to find a better solution for the second patch.
Don't expect me to follow bugzilla too :|
I'll drop this series off patchworks then, resend when you have something..
Sure, I'm trying to contain the change to PCI tree only. Hopefully, it can go
through Bjorn's tree. If we fail, we are back to this change. I'll ping you in that case.
Quoted 2 lines
Jason
PATCH 2/2 IB/hfi1,PCI: Allow bus reset while probing
In reply to Dennis DalessandroCalling into the new API to reset the secondary bus results in a deadlock. This occurs because the device/bus is already locked at probe time. Reverting back to the old behavior while the API is improved.
--- drivers/infiniband/hw/hfi1/pcie.c | 11 ++++------- drivers/pci/pci.c | 1 + include/linux/pci.h | 3 +++ 3 files changed, 8 insertions(+), 7 deletions(-)
drivers/infiniband/hw/hfi1/pcie.c +4 -7
@@ -893,14 +893,11 @@ static int trigger_sbr(struct hfi1_devdata *dd) }/* -* A secondary bus reset (SBR) issues a hot reset to our device. -* The following routine does a 1s wait after the reset is dropped -* per PCI Trhfa (recovery time). PCIe 3.0 section 6.6.1 - -* Conventional Reset, paragraph 3, line 35 also says that a 1s -* delay after a reset is required. Per spec requirements, -* the link is either working or not after that point. +* This is an end around to do an SBR during probe time. A new API needs +* to be implemented to have cleaner interface but this fixes the +* current brokenness */ -return pci_reset_bus (dev); +return pci_bridge_secondary_bus_reset (dev->bus->self); }/*
drivers/pci/pci.c +1 -0
@@ -4547,6 +4547,7 @@ int pci_bridge_secondary_bus_reset(struct pci_dev *dev)return pci_dev_wait (dev,"bus reset" ,PCIE_RESET_READY_POLL_MS ); } +EXPORT_SYMBOL_GPL (pci_bridge_secondary_bus_reset);static int pci_parent_bus_reset (struct pci_dev *dev,int probe) {
include/linux/pci.h +3 -0
@@ -1235,6 +1235,9 @@ void pci_bus_add_resource(struct pci_bus *bus, struct resource *res,int devm_request_pci_bus_resources (struct device *dev,struct list_head *resources); +/* Temporary until new and working PCI SBR API in place */ +int pci_bridge_secondary_bus_reset (struct pci_dev *dev); +#define pci_bus_for_each_resource (bus, res, i) \ for (i = 0; \ (res = pci_bus_resource_n(bus, i)) || i < PCI_BRIDGE_RESOURCE_NUM; \
Bjorn Helgaas
AppliedQuoted 32 lines · Dennis Dalessandro
Hi Bjorn, Doug and Jason,
As discussed on the pci list [1] our driver was broken with the following three changes in 4.19 merge window:
c6a44ba950d1("PCI: Renamepci_try_reset_bus()topci_reset_bus()")409888e0966e("IB/hfi1: Usepci_try_reset_bus()for initiating PCI Secondary Bus Reset")811c5cb37df4("PCI: Unify try slot and bus reset API")Ideally I'd like to see those patches reverted but we could also go with something like the following series. I know there is a desire to have a more clean API but since this is already the rc phase I think that is best delayed. Clearly the first attempt at an API clean up did not work out so well.
I'm not sure how you maintainers want to handle this since the fix straddles both subsystems so I'm sending it to both linux-rdma and linux-pci.
At the request of discussion on the linux-pci list I have also filed a bug [2].
[1] https://marc.info/?l=linux-pci&m=153539692917785&w=2
[2] https://bugzilla.kernel.org/show_bug.cgi?id=200985Thanks
---
Dennis Dalessandro (2):
PCI: Fix faulty logic inpci_reset_bus()
IB/hfi1,PCI: Allow bus reset while probing
Quoted 4 lines
drivers/infiniband/hw/hfi1/pcie.c | 11 ++++------- drivers/pci/pci.c | 3 ++- include/linux/pci.h | 3 +++ 3 files changed, 9 insertions(+), 8 deletions(-)
I applied both of these to for-linus for v4.19, thanks!