Skip to contentSkip to conversation

Documentation: PCI: pci-error-recovery: rearrange the general sequence

PATCH In review Wesley Sheng 3 participants 5 messages Jun 18, 2021 – Jul 2, 2021

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Documentation: PCI: pci-error-recovery: rearrange the general sequence

Reset_link() callback function was called before mmio_enabled() in pcie_do_recovery() function actually, so rearrange the general sequence betwen step 2 and step 3 accordingly.

---
 Documentation/PCI/pci-error-recovery.rst | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)
Documentation/PCI/pci-error-recovery.rst +12 -11
index 187f43a03200..ac6a8729ef28 100644
@@ -184,7 +184,14 @@ is STEP 6 (Permanent Failure).
    and prints an error to syslog.  A reboot is then required to
    get the device working again.
 
-STEP 2: MMIO Enabled
+STEP 2: Link Reset
+------------------
+The platform resets the link.  This is a PCI-Express specific step
+and is done whenever a fatal error has been detected that can be
+"solved" by resetting the link.
+
+
+STEP 3: MMIO Enabled
 --------------------
 The platform re-enables MMIO to the device (but typically not the
 DMA), and then calls the mmio_enabled() callback on all affected
@@ -197,8 +204,8 @@ information, if any, and eventually do things like trigger a device local
 reset or some such, but not restart operations. This callback is made if
 all drivers on a segment agree that they can try to recover and if no automatic
 link reset was performed by the HW. If the platform can't just re-enable IOs
-without a slot reset or a link reset, it will not call this callback, and
-instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset)
+without a slot reset, it will not call this callback, and
+instead will have gone directly or STEP 4 (Slot Reset)
 
 .. note::
 
@@ -210,7 +217,7 @@ instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset)
    such an error might cause IOs to be re-blocked for the whole
    segment, and thus invalidate the recovery that other devices
    on the same segment might have done, forcing the whole segment
-   into one of the next states, that is, link reset or slot reset.
+   into next states, that is, slot reset.
 
 The driver should return one of the following result codes:
   - PCI_ERS_RESULT_RECOVERED
@@ -233,17 +240,11 @@ The driver should return one of the following result codes:
 
 The next step taken depends on the results returned by the drivers.
 If all drivers returned PCI_ERS_RESULT_RECOVERED, then the platform
-proceeds to either STEP3 (Link Reset) or to STEP 5 (Resume Operations).
+proceeds to STEP 5 (Resume Operations).
 
 If any driver returned PCI_ERS_RESULT_NEED_RESET, then the platform
 proceeds to STEP 4 (Slot Reset)
 
-STEP 3: Link Reset
-------------------
-The platform resets the link.  This is a PCI-Express specific step
-and is done whenever a fatal error has been detected that can be
-"solved" by resetting the link.
-
 STEP 4: Slot Reset
 ------------------
 
Signature · 1 line
2.25.1
#1 lore raw
Quoted 4 lines · Wesley Sheng

Reset_link() callback function was called before mmio_enabled() in pcie_do_recovery() function actually, so rearrange the general sequence betwen step 2 and step 3 accordingly.

I don't think this is true in all cases. If pcie_do_recovery() is called with state==pci_channel_io_normal (i.e. non-fatal AER) the link won't be reset. EEH (ppc PCI error recovery thing) also uses .mmio_enabled() as described.

#2 lore raw
Quoted 10 lines · Oliver O'Halloran
Quoted 4 lines · Wesley Sheng

Reset_link() callback function was called before mmio_enabled() in pcie_do_recovery() function actually, so rearrange the general sequence betwen step 2 and step 3 accordingly.

I don't think this is true in all cases. If pcie_do_recovery() is called with state==pci_channel_io_normal (i.e. non-fatal AER) the link won't be reset. EEH (ppc PCI error recovery thing) also uses .mmio_enabled() as described.

Yes, in case of non-fatal AER, reset_link() callback (aer_root_reset() for AER and dpc_reset_link() for DPC) will not be invoked. And if .error_detected() return PCI_ERS_RESULT_CAN_RECOVER, .mmio_enabled() be called followed.

But if pcie_do_recovery() is called with state == pci_channel_io_frozen, reset_link() callback is called after .error_detected() but before .mmio_enabled(). So I thought Step 2: MMIO Enabled and Step 3: Link Reset should rearrange their sequence.

#3 lore raw
In reply to Wesley Sheng

Please make the subject a little more specific. "rearrange the general sequence" doesn't say anything about what was affected.

Quoted 3 lines · Wesley Sheng

Reset_link() callback function was called before mmio_enabled() in pcie_do_recovery() function actually, so rearrange the general sequence betwen step 2 and step 3 accordingly.

s/betwen/between/

Not sure "general" adds anything in this sentence. "Step 2 and step 3" are not meaningful here in the commit log. It needs to spell out what those steps are so the log makes sense by itself.

"reset_link" does not appear in pcie_do_recovery(). I'm guessing you're referring to the "reset_subordinates" function pointer?

Quoted 1 line

I didn't quite understand your response to Oliver, so I'll wait for your corrections and his ack before proceeding.

Quoted 25 lines
---
 Documentation/PCI/pci-error-recovery.rst | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)
Documentation/PCI/pci-error-recovery.rst +8 -1
index 187f43a03200..ac6a8729ef28 100644
@@ -184,7 +184,14 @@ is STEP 6 (Permanent Failure).
    and prints an error to syslog.  A reboot is then required to
    get the device working again.
 
-STEP 2: MMIO Enabled
+STEP 2: Link Reset
+------------------
+The platform resets the link.  This is a PCI-Express specific step
+and is done whenever a fatal error has been detected that can be
+"solved" by resetting the link.
+
+
+STEP 3: MMIO Enabled
 --------------------
 The platform re-enables MMIO to the device (but typically not the
 DMA), and then calls the mmio_enabled() callback on all affected

Documentation/PCI/pci-error-recovery.rst information, if any, and eventually do things like trigger a device local · line 208

Quoted 7 lines
Documentation/PCI/pci-error-recovery.rst +2 -2
@@ -197,8 +204,8 @@ information, if any, and eventually do things like trigger a device local
 reset or some such, but not restart operations. This callback is made if
 all drivers on a segment agree that they can try to recover and if no automatic
 link reset was performed by the HW. If the platform can't just re-enable IOs
-without a slot reset or a link reset, it will not call this callback, and
-instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset)
+without a slot reset, it will not call this callback, and
+instead will have gone directly or STEP 4 (Slot Reset)

s/or/to/ ?

Documentation/PCI/pci-error-recovery.rst instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset) · line 220

Quoted 8 lines
Documentation/PCI/pci-error-recovery.rst +1 -1
 .. note::
 
@@ -210,7 +217,7 @@ instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset)
    such an error might cause IOs to be re-blocked for the whole
    segment, and thus invalidate the recovery that other devices
    on the same segment might have done, forcing the whole segment
-   into one of the next states, that is, link reset or slot reset.
+   into next states, that is, slot reset.

s/into next states/into the next state/ ?

Quoted 24 lines
Documentation/PCI/pci-error-recovery.rst +1 -7
 The driver should return one of the following result codes:
   - PCI_ERS_RESULT_RECOVERED
@@ -233,17 +240,11 @@ The driver should return one of the following result codes:
 
 The next step taken depends on the results returned by the drivers.
 If all drivers returned PCI_ERS_RESULT_RECOVERED, then the platform
-proceeds to either STEP3 (Link Reset) or to STEP 5 (Resume Operations).
+proceeds to STEP 5 (Resume Operations).
 
 If any driver returned PCI_ERS_RESULT_NEED_RESET, then the platform
 proceeds to STEP 4 (Slot Reset)
 
-STEP 3: Link Reset
-------------------
-The platform resets the link.  This is a PCI-Express specific step
-and is done whenever a fatal error has been detected that can be
-"solved" by resetting the link.
-
 STEP 4: Slot Reset
 ------------------
 
Signature · 1 line
2.25.1
#4 lore raw
Quoted 8 lines · Bjorn Helgaas

Please make the subject a little more specific. "rearrange the general sequence" doesn't say anything about what was affected.

Quoted 3 lines · Wesley Sheng

Reset_link() callback function was called before mmio_enabled() in pcie_do_recovery() function actually, so rearrange the general sequence betwen step 2 and step 3 accordingly.

Quoted 9 lines

s/betwen/between/

Not sure "general" adds anything in this sentence. "Step 2 and step 3" are not meaningful here in the commit log. It needs to spell out what those steps are so the log makes sense by itself.

"reset_link" does not appear in pcie_do_recovery(). I'm guessing you're referring to the "reset_subordinates" function pointer?

Yes, you are right.
pcieaer-howto.rst has a section named with "Provide callbacks", the callback supplied to pcie_do_recovery() was referred to reset_link.

Quoted 5 lines
Quoted 1 line

I didn't quite understand your response to Oliver, so I'll wait for your corrections and his ack before proceeding.

OK.
I thought step 2 MMIO Enabled and step 3 link reset should swap sequence.

Quoted 70 lines
Quoted 25 lines
---
 Documentation/PCI/pci-error-recovery.rst | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)
Documentation/PCI/pci-error-recovery.rst +8 -1
index 187f43a03200..ac6a8729ef28 100644
@@ -184,7 +184,14 @@ is STEP 6 (Permanent Failure).
    and prints an error to syslog.  A reboot is then required to
    get the device working again.
 
-STEP 2: MMIO Enabled
+STEP 2: Link Reset
+------------------
+The platform resets the link.  This is a PCI-Express specific step
+and is done whenever a fatal error has been detected that can be
+"solved" by resetting the link.
+
+
+STEP 3: MMIO Enabled
 --------------------
 The platform re-enables MMIO to the device (but typically not the
 DMA), and then calls the mmio_enabled() callback on all affected

Documentation/PCI/pci-error-recovery.rst information, if any, and eventually do things like trigger a device local · line 208

Quoted 7 lines
Documentation/PCI/pci-error-recovery.rst +2 -2
@@ -197,8 +204,8 @@ information, if any, and eventually do things like trigger a device local
 reset or some such, but not restart operations. This callback is made if
 all drivers on a segment agree that they can try to recover and if no automatic
 link reset was performed by the HW. If the platform can't just re-enable IOs
-without a slot reset or a link reset, it will not call this callback, and
-instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset)
+without a slot reset, it will not call this callback, and
+instead will have gone directly or STEP 4 (Slot Reset)

s/or/to/ ?

Documentation/PCI/pci-error-recovery.rst instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset) · line 220

Quoted 8 lines
Documentation/PCI/pci-error-recovery.rst +1 -1
 .. note::
 
@@ -210,7 +217,7 @@ instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset)
    such an error might cause IOs to be re-blocked for the whole
    segment, and thus invalidate the recovery that other devices
    on the same segment might have done, forcing the whole segment
-   into one of the next states, that is, link reset or slot reset.
+   into next states, that is, slot reset.

s/into next states/into the next state/ ?

Quoted 24 lines
Documentation/PCI/pci-error-recovery.rst +1 -7
 The driver should return one of the following result codes:
   - PCI_ERS_RESULT_RECOVERED
@@ -233,17 +240,11 @@ The driver should return one of the following result codes:
 
 The next step taken depends on the results returned by the drivers.
 If all drivers returned PCI_ERS_RESULT_RECOVERED, then the platform
-proceeds to either STEP3 (Link Reset) or to STEP 5 (Resume Operations).
+proceeds to STEP 5 (Resume Operations).
 
 If any driver returned PCI_ERS_RESULT_NEED_RESET, then the platform
 proceeds to STEP 4 (Slot Reset)
 
-STEP 3: Link Reset
-------------------
-The platform resets the link.  This is a PCI-Express specific step
-and is done whenever a fatal error has been detected that can be
-"solved" by resetting the link.
-
 STEP 4: Slot Reset
 ------------------
 
Signature · 1 line
2.25.1