Skip to contentSkip to conversation

AER: aer_root_reset() non-native handling

PATCH Superseded Sean V Kelley 2 participants 3 messages Oct 30, 2020 – Nov 3, 2020

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 AER: aer_root_reset() non-native handling

If an OS has not been granted AER control via _OSC, then the OS should not make changes to PCI_ERR_ROOT_COMMAND and PCI_ERR_ROOT_STATUS related registers. Per section 4.5.1 of the System Firmware Intermediary (SFI) _OSC and DPC Updates ECN [1], this bit also covers these aspects of the PCI Express Advanced Error Reporting. Further, the handling of clear/enable of PCI_ERROR_ROOT_COMMAND when wrapped around PCI_ERR_ROOT_STATUS should have no effect and be removed. Based on the above and earlier discussion [2], make the following changes:

Add a check for the native case (i.e., AER control via _OSC) Re-order and remove some of the handling:
- clear PCI_ERR_ROOT_COMMAND ROOT_PORT_INTR_ON_MESG_MASK
- do reset
- clear PCI_ERR_ROOT_STATUS
- enable PCI_ERR_ROOT_COMMAND ROOT_PORT_INTR_ON_MESG_MASK

to this:

- clear PCI_ERR_ROOT_STATUS
- do reset

The current "clear, reset, enable" order suggests that the reset might cause errors that we should ignore. But I am unable to find a reference and the clearing of PCI_ERR_ROOT_STATUS does not require them.

[1] System Firmware Intermediary (SFI) _OSC and DPC Updates ECN, Feb 24, 2020, affecting PCI Firmware Specification, Rev. 3.2
https://members.pcisig.com/wg/PCI-SIG/document/14076
[2] https://lore.kernel.org/linux-pci/20201020162820.GA370938@bjorn-Precision-5520/

---
 drivers/pci/pcie/aer.c | 21 ++++++---------------
 1 file changed, 6 insertions(+), 15 deletions(-)
drivers/pci/pcie/aer.c +6 -15
index 65dff5f3457a..bbfb07842d89 100644
@@ -1361,23 +1361,14 @@ static pci_ers_result_t aer_root_reset(struct pci_dev *dev)
 	u32 reg32;
 	int rc;
 
-
-	/* Disable Root's interrupt in response to error messages */
-	pci_read_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, &reg32);
-	reg32 &= ~ROOT_PORT_INTR_ON_MESG_MASK;
-	pci_write_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, reg32);
+	if (pcie_aer_is_native(dev)) {
+		/* Clear Root Error Status */
+		pci_read_config_dword(dev, aer + PCI_ERR_ROOT_STATUS, &reg32);
+		pci_write_config_dword(dev, aer + PCI_ERR_ROOT_STATUS, reg32);
+	}
 
 	rc = pci_bus_error_reset(dev);
-	pci_info(dev, "Root Port link has been reset\n");
-
-	/* Clear Root Error Status */
-	pci_read_config_dword(dev, aer + PCI_ERR_ROOT_STATUS, &reg32);
-	pci_write_config_dword(dev, aer + PCI_ERR_ROOT_STATUS, reg32);
-
-	/* Enable Root Port's interrupt in response to error messages */
-	pci_read_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, &reg32);
-	reg32 |= ROOT_PORT_INTR_ON_MESG_MASK;
-	pci_write_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, reg32);
+	pci_info(dev, "Root Port link has been reset (%d)\n", rc);
 
 	return rc ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED;
 }
Signature · 1 line
2.29.2
#1 lore raw
Quoted 43 lines · Sean V Kelley

If an OS has not been granted AER control via _OSC, then the OS should not make changes to PCI_ERR_ROOT_COMMAND and PCI_ERR_ROOT_STATUS related registers. Per section 4.5.1 of the System Firmware Intermediary (SFI) _OSC and DPC Updates ECN [1], this bit also covers these aspects of the PCI Express Advanced Error Reporting. Further, the handling of clear/enable of PCI_ERROR_ROOT_COMMAND when wrapped around PCI_ERR_ROOT_STATUS should have no effect and be removed. Based on the above and earlier discussion [2], make the following changes:

Add a check for the native case (i.e., AER control via _OSC) Re-order and remove some of the handling:
- clear PCI_ERR_ROOT_COMMAND ROOT_PORT_INTR_ON_MESG_MASK
- do reset
- clear PCI_ERR_ROOT_STATUS
- enable PCI_ERR_ROOT_COMMAND ROOT_PORT_INTR_ON_MESG_MASK

to this:

- clear PCI_ERR_ROOT_STATUS
- do reset

The current "clear, reset, enable" order suggests that the reset might cause errors that we should ignore. But I am unable to find a reference and the clearing of PCI_ERR_ROOT_STATUS does not require them.

[1] System Firmware Intermediary (SFI) _OSC and DPC Updates ECN, Feb 24, 2020, affecting PCI Firmware Specification, Rev. 3.2
https://members.pcisig.com/wg/PCI-SIG/document/14076
[2] https://lore.kernel.org/linux-pci/20201020162820.GA370938@bjorn-Precision-5520/

---
 drivers/pci/pcie/aer.c | 21 ++++++---------------
 1 file changed, 6 insertions(+), 15 deletions(-)
drivers/pci/pcie/aer.c +0 -0
index 65dff5f3457a..bbfb07842d89 100644

drivers/pci/pcie/aer.c static pci_ers_result_t aer_root_reset(struct pci_dev *dev) · line 1368

Quoted 8 lines
drivers/pci/pcie/aer.c +0 -5
@@ -1361,23 +1361,14 @@ static pci_ers_result_t aer_root_reset(struct pci_dev *dev)
        u32 reg32;
        int rc;
 
-
-       /* Disable Root's interrupt in response to error messages */
-       pci_read_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, &reg32);
-       reg32 &= ~ROOT_PORT_INTR_ON_MESG_MASK;
-       pci_write_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, reg32);

There may be some reasons to disable interrupt first and then do resetting, clear status and re-enable interrupt.
Perhaps to avoid error noise, what would happen if the resetting causes errors itself ?

Thanks,
Ethan

Quoted 24 lines
drivers/pci/pcie/aer.c +6 -11
+       if (pcie_aer_is_native(dev)) {
+               /* Clear Root Error Status */
+               pci_read_config_dword(dev, aer + PCI_ERR_ROOT_STATUS, &reg32);
+               pci_write_config_dword(dev, aer + PCI_ERR_ROOT_STATUS, reg32);
+       }
 
        rc = pci_bus_error_reset(dev);
-       pci_info(dev, "Root Port link has been reset\n");
-
-       /* Clear Root Error Status */
-       pci_read_config_dword(dev, aer + PCI_ERR_ROOT_STATUS, &reg32);
-       pci_write_config_dword(dev, aer + PCI_ERR_ROOT_STATUS, reg32);
-
-       /* Enable Root Port's interrupt in response to error messages */
-       pci_read_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, &reg32);
-       reg32 |= ROOT_PORT_INTR_ON_MESG_MASK;
-       pci_write_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, reg32);
+       pci_info(dev, "Root Port link has been reset (%d)\n", rc);
 
        return rc ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED;
 }
--

2.29.2

#2 lore raw
Quoted 48 lines · Ethan Zhao
Quoted 47 lines · Sean V Kelley

If an OS has not been granted AER control via _OSC, then the OS should not make changes to PCI_ERR_ROOT_COMMAND and PCI_ERR_ROOT_STATUS related registers. Per section 4.5.1 of the System Firmware Intermediary (SFI) _OSC and DPC Updates ECN [1], this bit also covers these aspects of the PCI Express Advanced Error Reporting. Further, the handling of clear/enable of PCI_ERROR_ROOT_COMMAND when wrapped around PCI_ERR_ROOT_STATUS should have no effect and be removed. Based on the above and earlier discussion [2], make the following changes:

Add a check for the native case (i.e., AER control via _OSC) Re-order and remove some of the handling:
- clear PCI_ERR_ROOT_COMMAND ROOT_PORT_INTR_ON_MESG_MASK
- do reset
- clear PCI_ERR_ROOT_STATUS
- enable PCI_ERR_ROOT_COMMAND ROOT_PORT_INTR_ON_MESG_MASK

to this:

- clear PCI_ERR_ROOT_STATUS
- do reset

The current "clear, reset, enable" order suggests that the reset might cause errors that we should ignore. But I am unable to find a reference and the clearing of PCI_ERR_ROOT_STATUS does not require them.

[1] System Firmware Intermediary (SFI) _OSC and DPC Updates ECN, Feb 24, 2020, affecting PCI Firmware Specification, Rev. 3.2
https://members.pcisig.com/wg/PCI-SIG/document/14076
[2] https://lore.kernel.org/linux-pci/20201020162820.GA370938@bjorn-Precision-5520/

---
 drivers/pci/pcie/aer.c | 21 ++++++---------------
 1 file changed, 6 insertions(+), 15 deletions(-)
drivers/pci/pcie/aer.c +0 -0
index 65dff5f3457a..bbfb07842d89 100644
@@ -1361,23 +1361,14 @@ static pci_ers_result_t aer_root_reset(struct 
 pci_dev *dev)
        u32 reg32;
        int rc;
 
Quoted 11 lines

drivers/pci/pcie/aer.c static pci_ers_result_t aer_root_reset(struct · line 1366

Quoted 7 lines
drivers/pci/pcie/aer.c +0 -5
-
-       /* Disable Root's interrupt in response to error messages */
-       pci_read_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, 
 &reg32);
-       reg32 &= ~ROOT_PORT_INTR_ON_MESG_MASK;
-       pci_write_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, 
 reg32);

There may be some reasons to disable interrupt first and then do resetting, clear status and re-enable interrupt.
Perhaps to avoid error noise, what would happen if the resetting causes errors itself ?

I don’t have the historical context, but in looking at pci_bus_reset() it’s clear that it will probe slots for reset capability and then fall back to secondary bus reset. It’s possible that an attempt to reset a slot could result in an error on some hardware. This would potentially result in repeated error reporting. Not having the historical context, it’s best to leave in the clear/re-enabling of the interrupt when performing the reset.

Sean

Quoted 36 lines

Thanks,
Ethan

Quoted 32 lines
drivers/pci/pcie/aer.c +5 -4
+       if (pcie_aer_is_native(dev)) {
+               /* Clear Root Error Status */
+               pci_read_config_dword(dev, aer + PCI_ERR_ROOT_STATUS, 
 &reg32);
+               pci_write_config_dword(dev, aer + 
 PCI_ERR_ROOT_STATUS, reg32);
+       }
 
        rc = pci_bus_error_reset(dev);
-       pci_info(dev, "Root Port link has been reset\n");
-
-       /* Clear Root Error Status */
-       pci_read_config_dword(dev, aer + PCI_ERR_ROOT_STATUS, 

&reg32);
- pci_write_config_dword(dev, aer + PCI_ERR_ROOT_STATUS,
reg32);

drivers/pci/pcie/aer.c +0 -2
-
-       /* Enable Root Port's interrupt in response to error messages 

*/
- pci_read_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND,
&reg32);

drivers/pci/pcie/aer.c +0 -2
-       reg32 |= ROOT_PORT_INTR_ON_MESG_MASK;
-       pci_write_config_dword(dev, aer + PCI_ERR_ROOT_COMMAND, 

reg32);
+ pci_info(dev, "Root Port link has been reset (%d)\n", rc);

return rc ? PCI_ERS_RESULT_DISCONNECT :
PCI_ERS_RESULT_RECOVERED;
}
--
2.29.2