PCI/AER: Handle Advisory Non-Fatal error
Subsystems (4)
v5 cover PCI/AER: Handle Advisory Non-Fatal error
Hi,
This is a relay work of Qingshun's v2 [1], but changed to focus on ANFE processing as subject suggests and drops trace-event for now. I think it's a bit heavy to do extra IOes to get PCIe registers only for trace purpose and not see it a community request for now.
According to PCIe Base Specification Revision 6.1, Sections 6.2.3.2.4 and 6.2.4.3, certain uncorrectable errors will signal ERR_COR instead of ERR_NONFATAL, logged as Advisory Non-Fatal Error(ANFE), and set bits in both Correctable Error(CE) Status register and Uncorrectable Error(UE) Status register. Currently, when handling AER events the kernel will only look at CE status or UE status, but never both. In the ANFE case, bits set in the UE status register will not be reported and cleared until the next FE/NFE arrives.
For instance, previously, when the kernel receives an ANFE with Poisoned TLP in OS native AER mode, only the status of CE will be reported and cleared:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00002000/00000000
[13] NonFatalErrIf the kernel receives a Malformed TLP after that, two UEs will be reported, which is unexpected. The Malformed TLP Header is lost since the previous ANFE gated the TLP header logs:
PCIe Bus Error: severity="Uncorrectable (Fatal), type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00041000/00180020
[12] TLP (First)
[18] MalfTLPTo handle this case properly, calculate potential ANFE related status bits and save in aer_err_info. Use this information to determine the status bits that need to be cleared.
Now, for the previous scenario, both CE status and related UE status will be reported and cleared after ANFE:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00002000/00000000
[13] NonFatalErr
Uncorrectable errors that may cause Advisory Non-Fatal:
[12] TLPNote:
checkpatch.pl will produce following warnings on PATCH1&2:
WARNING: 'UE' may be misspelled - perhaps 'USE'?
#22:
uncorrectable error(UE) status should be cleared. However, there is no
...similar warnings omitted...
This is a false-positive, so not fixed.
WARNING: Prefer a maximum 75 chars per line (possible unwrapped commit description?) #10:
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
...similar warnings omitted...
For readability reasons, these warnings are not fixed.
[1] https://lore.kernel.org/linux-pci/[email protected]
Thanks
Qingshun, Zhenzhong
Changelog:
v5:
- squash patch 1 and 3 (Kuppuswamy)
- add comment about avoiding race and fix typo error (Kuppuswamy)
- collect Jonathan and Kuppuswamy's R-b
v4:
- Fix a race in anfe_get_uc_status() (Jonathan)
- Add a comment to explain side effect of processing ANFE as NFE (Jonathan)
- Drop the check for PCI_EXP_DEVSTA_NFED
v3:
- Split ANFE print and processing to two patches (Bjorn)
- Simplify ANFE handling, drop trace event
- Polish comments and patch description
- Add Tested-by
v2:
- Reference to the latest PCIe Specification in both commit messages and comments, as suggested by Bjorn Helgaas.
- Describe the reason for storing additional information in aer_err_info in the commit message of PATCH 1, as suggested by Bjorn Helgaas.
- Add more details of behavior changes in the commit message of PATCH 2, as suggested by Bjorn Helgaas.
v4: https://lkml.org/lkml/2024/5/9/247
v3: https://lore.kernel.org/lkml/[email protected]
v2: https://lore.kernel.org/linux-pci/[email protected]
v1: https://lore.kernel.org/linux-pci/[email protected]
Zhenzhong Duan (2):
PCI/AER: Clear UNCOR_STATUS bits that might be ANFE PCI/AER: Print UNCOR_STATUS bits that might be ANFE
drivers/pci/pci.h | 1 + drivers/pci/pcie/aer.c | 79 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 79 insertions(+), 1 deletion(-)
Signature · 1 line
2.34.1
PATCH v5 1/2 PCI/AER: Clear UNCOR_STATUS bits that might be ANFE
In some cases the detector of a Non-Fatal Error(NFE) is not the most appropriate agent to determine the type of the error. For example, when software performs a configuration read from a non-existent device or Function, completer will send an ERR_NONFATAL Message. On some platforms, ERR_NONFATAL results in a System Error, which breaks normal software probing.
Advisory Non-Fatal Error(ANFE) is a special case that can be used in above scenario. It is predominantly determined by the role of the detecting agent (Requester, Completer, or Receiver) and the specific error. In such cases, an agent with AER signals the NFE (if enabled) by sending an ERR_COR Message as an advisory to software, instead of sending ERR_NONFATAL.
When processing an ANFE, ideally both correctable error(CE) status and uncorrectable error(UE) status should be cleared. However, there is no way to fully identify the UE associated with ANFE. Even worse, Non-Fatal Error(NFE) may set the same UE status bit as ANFE. Treating an ANFE as NFE will reproduce above mentioned issue, i.e., breaking software probing; treating NFE as ANFE will make us ignore some UEs which need active recover operation. To avoid clearing UEs that are not ANFE by accident, the most conservative route is taken here: If any of the NFE Detected bits is set in Device Status, do not touch UE status, they should be cleared later by the UE handler. Otherwise, a specific set of UEs that may be raised as ANFE according to the PCIe specification will be cleared if their corresponding severity is Non-Fatal.
To achieve above purpose, cache UNCOR_STATUS bits that might be ANFE in aer_err_info.anfe_status and clean them in pci_aer_handle_error(). aer_err_info.anfe_status will also be used to print ANFE related bits in following patch.
For instance, previously, when the kernel receives an ANFE with Poisoned TLP in OS native AER mode, only the status of CE will be reported and cleared:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00002000/00000000
[13] NonFatalErrIf the kernel receives a Malformed TLP after that, two UEs will be reported, which is unexpected. The Malformed TLP Header is lost since the previous ANFE gated the TLP header logs:
PCIe Bus Error: severity="Uncorrectable (Fatal), type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00041000/00180020
[12] TLP (First)
[18] MalfTLPTo handle this case properly, calculate potential ANFE related status bits and save in aer_err_info. Use this information to determine the status bits that need to be cleared.
Now, for the previous scenario, both CE status and related UE status will be cleared after ANFE:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00002000/00000000
[13] NonFatalErr PCIe Bus Error: severity="Uncorrectable (Fatal), type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00040000/00180020
[18] MalfTLP (First)
--- drivers/pci/pci.h | 1 + drivers/pci/pcie/aer.c | 64 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 64 insertions(+), 1 deletion(-)
drivers/pci/pci.h +1 -0
@@ -410,6 +410,7 @@ struct aer_err_info {unsigned int status;/* COR/UNCOR Error Status */ unsigned int mask;/* COR/UNCOR Error Mask */ +unsigned int anfe_status;/* UNCOR Error Status for ANFE */ struct pcie_tlp_log tlp;/* TLP Header */ };
drivers/pci/pcie/aer.c +63 -1
96 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v5 1/2] PCI/AER: Clear UNCOR_STATUS bits that might be ANFE
Quoted 66 lines · Zhenzhong Duan
In some cases the detector of a Non-Fatal Error(NFE) is not the most appropriate agent to determine the type of the error. For example, when software performs a configuration read from a non-existent device or Function, completer will send an
ERR_NONFATALMessage. On some platforms,ERR_NONFATALresults in a System Error, which breaks normal software probing.Advisory Non-Fatal Error(ANFE) is a special case that can be used in above scenario. It is predominantly determined by the role of the detecting agent (Requester, Completer, or Receiver) and the specific error. In such cases, an agent with AER signals the NFE (if enabled) by sending an
ERR_CORMessage as an advisory to software, instead of sendingERR_NONFATAL.When processing an ANFE, ideally both correctable error(CE) status and uncorrectable error(UE) status should be cleared. However, there is no way to fully identify the UE associated with ANFE. Even worse, Non-Fatal Error(NFE) may set the same UE status bit as ANFE. Treating an ANFE as NFE will reproduce above mentioned issue, i.e., breaking software probing; treating NFE as ANFE will make us ignore some UEs which need active recover operation. To avoid clearing UEs that are not ANFE by accident, the most conservative route is taken here: If any of the NFE Detected bits is set in Device Status, do not touch UE status, they should be cleared later by the UE handler. Otherwise, a specific set of UEs that may be raised as ANFE according to the PCIe specification will be cleared if their corresponding severity is Non-Fatal.
To achieve above purpose, cache
UNCOR_STATUSbits that might be ANFE in aer_err_info.anfe_status and clean them inpci_aer_handle_error(). aer_err_info.anfe_status will also be used to print ANFE related bits in following patch.For instance, previously, when the kernel receives an ANFE with Poisoned TLP in OS native AER mode, only the status of CE will be reported and cleared:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)device [8086:0db0] error status/mask=00002000/00000000 [13] NonFatalErrIf the kernel receives a Malformed TLP after that, two UEs will be reported, which is unexpected. The Malformed TLP Header is lost since the previous ANFE gated the TLP header logs:
PCIe Bus Error: severity="Uncorrectable (Fatal), type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00041000/00180020 [12] TLP (First) [18] MalfTLPTo handle this case properly, calculate potential ANFE related status bits and save in
aer_err_info. Use this information to determine the status bits that need to be cleared.Now, for the previous scenario, both CE status and related UE status will be cleared after ANFE:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)device [8086:0db0] error status/mask=00002000/00000000 [13] NonFatalErrPCIe Bus Error: severity="Uncorrectable (Fatal), type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00040000/00180020
[18] MalfTLP (First)
Quoted 6 lines
Tested-by Yudong Wang <[email protected]>Co-developed-by "Wang, Qingshun" <[email protected]>Signed-off-by "Wang, Qingshun" <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Kuppuswamy Sathyanarayanan <[email protected]>
This no longer applies cleanly; would you mind rebasing it to pci/main (v6.17-rc1)? There have been recent AER changes; if they affect the dmesg text, could you update that as well?
drivers/pci/pcie/aer.c void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn, · line 1219
Quoted 10 lines
drivers/pci/pcie/aer.c +10
-0+static void anfe_get_uc_status (struct pci_dev *dev,struct aer_err_info *info) +{ +u32 uncor_mask, uncor_status, anfe_status; +u16 device_status; +int aer = dev->aer_cap; + +/* +* To avoid race between device status read and error status register read, +* cache uncorrectable error status before checking for NFE in device status +* register.
I can't tell for sure from the patch, but if this doesn't fit in 80 columns, can you rewrap it so it matches the rest of the file?
Quoted 5 lines
drivers/pci/pcie/aer.c +5
-0+*/ +pci_read_config_dword (dev, aer +PCI_ERR_UNCOR_STATUS , &uncor_status); +pci_read_config_dword (dev, aer +PCI_ERR_UNCOR_MASK , &uncor_mask); +/* +* According to PCIe Base Specification Revision 6.1 Section 6.2.3.2.4,
drivers/pci/pcie/aer.c void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn, · line 1234
Quoted 10 lines
drivers/pci/pcie/aer.c +10
-0+* if an UNCOR error is raised as Advisory Non-Fatal error, it will +* match the following conditions: +* a. The severity of the error is Non-Fatal. +* b. The error is one of the following: +* 1. Poisoned TLP (Section 6.2.3.2.4.3) +* 2. Completion Timeout (Section 6.2.3.2.4.4) +* 3. Completer Abort (Section 6.2.3.2.4.1) +* 4. Unexpected Completion (Section 6.2.3.2.4.5) +* 5. Unsupported Request (Section 6.2.3.2.4.1) +*/
Could you update the citation to PCIe 7.0, please?
Bjorn
PATCH v5 2/2 PCI/AER: Print UNCOR_STATUS bits that might be ANFE
In reply to Zhenzhong DuanWhen an Advisory Non-Fatal error(ANFE) triggers, both correctable error(CE) status and ANFE related uncorrectable error(UE) status will be printed:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00002000/00000000
[13] NonFatalErr
Uncorrectable errors that may cause Advisory Non-Fatal:
[12] TLP--- drivers/pci/pcie/aer.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/pci/pcie/aer.c +15 -0
@@ -681,6 +681,7 @@ static void __aer_print_error(struct pci_dev *dev, {const char **strings;unsigned long status = info->status & ~info->mask; +unsigned long anfe_status = info->anfe_status;const char *level, *errmsg;int i;
@@ -701,6 +702,20 @@ static void __aer_print_error(struct pci_dev *dev, info->first_error == i ? " (First)" : ""); }pci_dev_aer_stats_incr (dev, info); + +if (!anfe_status) +return ; + + strings = aer_uncorrectable_error_string; +pci_printk (level, dev,"Uncorrectable errors that may cause Advisory Non-Fatal:\n" ); + +for_each_set_bit (i, &anfe_status,32 ) { + errmsg = strings[i]; +if (!errmsg) + errmsg ="Unknown Error Bit" ; + +pci_printk (level, dev," [%2d] %s\n" , i, errmsg); + } }void aer_print_error (struct pci_dev *dev,struct aer_err_info *info)
Signature · 1 line
2.34.1
Re: [PATCH v5 2/2] PCI/AER: Print UNCOR_STATUS bits that might be ANFE
Quoted 9 lines · Zhenzhong Duan
When an Advisory Non-Fatal error(ANFE) triggers, both correctable error(CE) status and ANFE related uncorrectable error(UE) status will be printed:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)device [8086:0db0] error status/mask=00002000/00000000 [13] NonFatalErr Uncorrectable errors that may cause Advisory Non-Fatal: [12] TLP
Forgot to mention on other patch, but please add spaces between the spelled-out terms and the "()" abbreviation, e.g., "Correctable Error (CE)".
Also, can you update this commit log to say what the patch does? It's OK if it repeats and/or expands on the subject.
Quoted 23 lines
Tested-by Yudong Wang <[email protected]>Co-developed-by "Wang, Qingshun" <[email protected]>Signed-off-by "Wang, Qingshun" <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Kuppuswamy Sathyanarayanan <[email protected]>--- drivers/pci/pcie/aer.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/pci/pcie/aer.c +1
-0@@ -681,6 +681,7 @@ static void __aer_print_error(struct pci_dev *dev, {const char **strings;unsigned long status = info->status & ~info->mask; +unsigned long anfe_status = info->anfe_status;const char *level, *errmsg;int i;
drivers/pci/pcie/aer.c static void __aer_print_error(struct pci_dev *dev, · line 710
Quoted 9 lines
drivers/pci/pcie/aer.c +6
-0@@ -701,6 +702,20 @@ static void __aer_print_error(struct pci_dev *dev, info->first_error == i ? " (First)" : ""); }pci_dev_aer_stats_incr (dev, info); + +if (!anfe_status) +return ; + + strings = aer_uncorrectable_error_string; +pci_printk (level, dev,"Uncorrectable errors that may cause Advisory Non-Fatal:\n" );
Will have to look at the spec more, but I don't think "may cause" is quite the right wording here. It's not that an Uncorrectable Error causes a separate Advisory Non-Fatal Error; IIUC there's only a single error and it's just *treated* and signaled differently.
drivers/pci/pcie/aer.c static void __aer_print_error(struct pci_dev *dev, · line 717
Quoted 7 lines
drivers/pci/pcie/aer.c +7
-0+ +for_each_set_bit (i, &anfe_status,32 ) { + errmsg = strings[i]; +if (!errmsg) + errmsg ="Unknown Error Bit" ; + +pci_printk (level, dev," [%2d] %s\n" , i, errmsg);
I think we might have removed pci_printk() recently, so this might need adjustment.
Quoted 7 lines
drivers/pci/pcie/aer.c +1
-0+ } }void aer_print_error (struct pci_dev *dev,struct aer_err_info *info)Signature · 1 line
2.34.1
Re: [PATCH v5 2/2] PCI/AER: Print UNCOR_STATUS bits that might be ANFE
In reply to Zhenzhong Duan[+cc Matt]
Quoted 33 lines · Zhenzhong Duan
When an Advisory Non-Fatal error(ANFE) triggers, both correctable error(CE) status and ANFE related uncorrectable error(UE) status will be printed:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)device [8086:0db0] error status/mask=00002000/00000000 [13] NonFatalErr Uncorrectable errors that may cause Advisory Non-Fatal: [12] TLPTested-by Yudong Wang <[email protected]>Co-developed-by "Wang, Qingshun" <[email protected]>Signed-off-by "Wang, Qingshun" <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Kuppuswamy Sathyanarayanan <[email protected]>--- drivers/pci/pcie/aer.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/pci/pcie/aer.c +1
-0@@ -681,6 +681,7 @@ static void __aer_print_error(struct pci_dev *dev, {const char **strings;unsigned long status = info->status & ~info->mask; +unsigned long anfe_status = info->anfe_status;const char *level, *errmsg;int i;
drivers/pci/pcie/aer.c static void __aer_print_error(struct pci_dev *dev, · line 707
Quoted 6 lines
drivers/pci/pcie/aer.c +3
-0@@ -701,6 +702,20 @@ static void __aer_print_error(struct pci_dev *dev, info->first_error == i ? " (First)" : ""); }pci_dev_aer_stats_incr (dev, info); + +if (!anfe_status) +return ;
__aer_print_error() is used by both native AER handling, where Linux fields the AER interrupt and reads the AER status registers directly, and APEI GHES firmware-first error handling, where platform firmware fields the AER interrupt, reads the AER status registers, and packages them up to hand off to Linux via aer_recover_queue().
But the previous patch only sets info->anfe_status for the native path, so the APEI GHES path doesn't get the benefit of this change.
I think both paths should log the same ANFE information.
Quoted 17 lines
drivers/pci/pcie/aer.c +11
-0+ + strings = aer_uncorrectable_error_string; +pci_printk (level, dev,"Uncorrectable errors that may cause Advisory Non-Fatal:\n" ); + +for_each_set_bit (i, &anfe_status,32 ) { + errmsg = strings[i]; +if (!errmsg) + errmsg ="Unknown Error Bit" ; + +pci_printk (level, dev," [%2d] %s\n" , i, errmsg); + } }void aer_print_error (struct pci_dev *dev,struct aer_err_info *info)Signature · 1 line
2.34.1
Hi Bjorn,
Kindly ping, this series got Reviewed-by and no comments for a month. Will you think about picking it or further improvements are needed. Look forward to your suggestions.
Thanks
Zhenzhong
Quoted 127 lines
-----Original Message-----
From: Duan, Zhenzhong <[email protected]>
Subject: [PATCH v5 0/2] PCI/AER: Handle Advisory Non-Fatal errorHi,
This is a relay work of Qingshun's v2 [1], but changed to focus on ANFE processing as subject suggests and drops trace-event for now. I think it's a bit heavy to do extra IOes to get PCIe registers only for trace purpose and not see it a community request for now.
According to PCIe Base Specification Revision 6.1, Sections 6.2.3.2.4 and 6.2.4.3, certain uncorrectable errors will signal
ERR_CORinstead ofERR_NONFATAL, logged as Advisory Non-Fatal Error(ANFE), and set bits in both Correctable Error(CE) Status register and Uncorrectable Error(UE) Status register. Currently, when handling AER events the kernel will only look at CE status or UE status, but never both. In the ANFE case, bits set in the UE status register will not be reported and cleared until the next FE/NFE arrives.For instance, previously, when the kernel receives an ANFE with Poisoned TLP in OS native AER mode, only the status of CE will be reported and cleared:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00002000/00000000
[13] NonFatalErrIf the kernel receives a Malformed TLP after that, two UEs will be reported, which is unexpected. The Malformed TLP Header is lost since the previous ANFE gated the TLP header logs:
PCIe Bus Error: severity="Uncorrectable (Fatal), type=Transaction Layer,
(Receiver ID)
device [8086:0db0] error status/mask=00041000/00180020
[12] TLP (First)
[18] MalfTLPTo handle this case properly, calculate potential ANFE related status bits and save in
aer_err_info. Use this information to determine the status bits that need to be cleared.Now, for the previous scenario, both CE status and related UE status will be reported and cleared after ANFE:
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00002000/00000000
[13] NonFatalErr
Uncorrectable errors that may cause Advisory Non-Fatal:[12] TLPNote:
checkpatch.pl will produce following warnings on PATCH1&2:WARNING: 'UE' may be misspelled - perhaps 'USE'?
#22:
uncorrectable error(UE) status should be cleared. However, there is no...similar warnings omitted...
This is a false-positive, so not fixed.
WARNING: Prefer a maximum 75 chars per line (possible unwrapped commit description?)
#10:
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)...similar warnings omitted...
For readability reasons, these warnings are not fixed.
[1] https://lore.kernel.org/linux-pci/20240125062802.50819-1-
[email protected]Thanks
Qingshun, ZhenzhongChangelog:
v5:
- squash patch 1 and 3 (Kuppuswamy)
- add comment about avoiding race and fix typo error (Kuppuswamy)
- collect Jonathan and Kuppuswamy's R-bv4:
- Fix a race inanfe_get_uc_status()(Jonathan)
- Add a comment to explain side effect of processing ANFE as NFE (Jonathan)
- Drop the check forPCI_EXP_DEVSTA_NFEDv3:
- Split ANFE print and processing to two patches (Bjorn)
- Simplify ANFE handling, drop trace event
- Polish comments and patch description
- Add Tested-byv2:
- Reference to the latest PCIe Specification in both commit messages and comments, as suggested by Bjorn Helgaas.
- Describe the reason for storing additional information inaer_err_infoin the commit message of PATCH 1, as suggested by Bjorn Helgaas.
- Add more details of behavior changes in the commit message of PATCH 2, as suggested by Bjorn Helgaas.v4: https://lkml.org/lkml/2024/5/9/247
v3: https://lore.kernel.org/lkml/20240417061407.1491361-1-
[email protected]
v2: https://lore.kernel.org/linux-pci/20240125062802.50819-1-
[email protected]
v1: https://lore.kernel.org/linux-pci/20240111073227.31488-1-
[email protected]Zhenzhong Duan (2):
PCI/AER: ClearUNCOR_STATUSbits that might be ANFE PCI/AER: PrintUNCOR_STATUSbits that might be ANFE
drivers/pci/pci.h| 1 +drivers/pci/pcie/aer.c| 79
+++++++++++++++++++++++++++++++++++++++++-2 files changed, 79 insertions(+), 1 deletion(-)--
2.34.1
v5 cover PCI/AER: Handle Advisory Non-Fatal error
Hello. My team had independently started to make a change similar to this before realizing that someone had already taken a stab at it. It is highly desirable in my mind to have an improved handling of Advisory Errors in the upstream kernel. Is there anything we can do to help move this effort along? Perhaps testing? We have a decent variety of system configurations & are able to inject various kinds of errors via special devices/commands etc.
Thanks,
-Matt
Hi Matthew,
Feel free to take it over if you are interested. Maintainer didn't respond to this series, perhaps he expects some improvement in the series.
Thanks
Zhenzhong
Quoted 13 lines
-----Original Message-----
From: Matthew W Carlis <[email protected]>
Subject: [PATCH v5 0/2] PCI/AER: Handle Advisory Non-Fatal errorHello. My team had independently started to make a change similar to this before realizing that someone had already taken a stab at it. It is highly desirable in my mind to have an improved handling of Advisory Errors in the upstream kernel. Is there anything we can do to help move this effort along? Perhaps testing? We have a decent variety of system configurations & are able to inject various kinds of errors via special devices/commands etc.
Thanks,
-Matt
Quoted 5 lines · Zhenzhong
Hi Matthew,
Feel free to take it over if you are interested. Maintainer didn't respond to this series, perhaps he expects some improvement in the series.
I'm terribly sorry, this is my fault. It just fell off my list for no good reason. Matthew, if you are able to test and/or provide a Reviewed-by, that would be the best thing you can do to move this forward (although neither is actually necessary).
Bjorn
Quoted 13 lines
Quoted 13 lines
-----Original Message-----
From: Matthew W Carlis <[email protected]>
Subject: [PATCH v5 0/2] PCI/AER: Handle Advisory Non-Fatal errorHello. My team had independently started to make a change similar to this before realizing that someone had already taken a stab at it. It is highly desirable in my mind to have an improved handling of Advisory Errors in the upstream kernel. Is there anything we can do to help move this effort along? Perhaps testing? We have a decent variety of system configurations & are able to inject various kinds of errors via special devices/commands etc.
Thanks,
-Matt
v5 cover PCI/AER: Handle Advisory Non-Fatal error
On Fri, 22 Aug 2025 11:51:12 -0500, Bjorn Helgaas wrote
Quoted 4 lines
I'm terribly sorry, this is my fault. It just fell off my list for no good reason. Matthew, if you are able to test and/or provide a Reviewed-by, that would be the best thing you can do to move this forward (although neither is actually necessary).
It seems for pci there is always a massive list of things in flight.. Difficult for any mortal to keep up with. We pulled the patch into our kernel & have started testing it. I'll sync-up with my team internally to see exactly what the plan is & how long we think it will take.
Cheers!
-Matt
Quoted 6 lines
-----Original Message-----
From: Matthew W Carlis <[email protected]>
Subject: [PATCH v5 0/2] PCI/AER: Handle Advisory Non-Fatal error
Quoted 8 lines
On Fri, 22 Aug 2025 11:51:12 -0500, Bjorn Helgaas wrote
Quoted 4 lines
I'm terribly sorry, this is my fault. It just fell off my list for no good reason. Matthew, if you are able to test and/or provide a Reviewed-by, that would be the best thing you can do to move this forward (although neither is actually necessary).
It seems for pci there is always a massive list of things in flight.. Difficult for any mortal to keep up with.
Fully agree, never mind, Bjorn.
BRs,
Zhenzhong
v5 cover PCI/AER: Handle Advisory Non-Fatal error
In reply to Bjorn HelgaasOn Fri, 22 Aug 2025 11:51:12 -0500, Bjorn Helgaas wrote
Quoted 2 lines
Matthew, if you are able to test and/or provide a Reviewed-by, that would be the best thing you can do to move this forward ...
I spent some time looking at the patch thinking about it a little more carefully. The only thing I don't really like in this revision of the patch is the logging for "may cause Advisory". Example below from "[PATCH v5 2/2] PCI/AER: Print UNCOR_STATUS bits that might be ANFE".
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00002000/00000000
[13] NonFatalErr
Uncorrectable errors that may cause Advisory Non-Fatal:
[12] TLP
I don't think we really need to log the UE caused by ANF any differently than any other UE & in fact I would prefer not to. In my mind we should log all the UE status bits via the same format as before. Taking from example above, in my mind it would be nice if the logging looked like this.
AER: Correctable error message received from 0000:b7:02.0
PCIe Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
device [8086:0db0] error status/mask=00002000/00000000
[13] NonFatalErr
PCIe Bus Error: severity=Uncorrectable (Non-Fatal), type=Transaction Layer
[12] TLP
If there was only one error (that triggered ANF handling) then we would know that the Non-Fatal UE was what triggered the NonFatalErr. If some other Non-Fatal errors are happening at the same time then it doesn't really matter which was sent via ERR_COR vs ERR_NONFATAL since we would also know from Root Error Status that we had received at least one of each message type. The objective in my mind being to free up header-logs & log status details without making error the recovery worse.
Does this sound reasonable or unreasonable? I can update the patch-set & re-submit if 'reasonable'.
Cheers!
-Matt