PCI/AER: Support Advisory Non-Fatal Errors
Subsystems (2)
cover PCI/AER: Support Advisory Non-Fatal Errors
PCIe r7.0 sec 6.2.4.3 specifies that certain Non-Fatal Errors may be signaled with ERR_COR instead of ERR_NONFATAL. These so-called Advisory Non-Fatal Errors are masked by default and hence not reported by Linux.
This series seeks to change that. The meat of it is in patch [6/6]. There have been several previous attempts to support Advisory Non-Fatal Errors which are all linked in the patch.
The preceding patches contain prep work. In particular, they ensure that the detecting agent and layer is reported correctly for each error (patch [1/6] and [2/6]), simplify logging of the error source (patch [3/6]), fix reporting of the TLP Prefix/Header Log (patch [4/6]) and move code around (patch [5/6]).
The patches have been tested internally, but further testing would be appreciated to raise the confidence. Thanks!
Lukas Wunner (6):
PCI/AER: Fix mapping of errors to agent & layer PCI/AER: Log agent & layer for each individual error PCI/AER: Deduplicate logging of Error Source Identification PCI/AER: Emit TLP Log only for unmasked errors PCI/AER: Move retrieval of FEP and TLP Log into helper PCI/AER: Support Advisory Non-Fatal Errors
drivers/pci/pci.h | 4 + drivers/pci/pcie/aer.c | 250 ++++++++++++++++++++++++++++++++--------- 2 files changed, 200 insertions(+), 54 deletions(-)
Signature · 1 line
2.53.0
PATCH 1/6 PCI/AER: Fix mapping of errors to agent & layer
PCIe r7.0 sec 6.2.7 documents the agent and layer of each Correctable and Uncorrectable Error. Based on this spec section, the AER driver maps detected errors to an agent and layer using a set of macros and logs them.
Most errors listed in sec 6.2.7 map to the "Receiver" agent and "Transaction Layer", so the macros use these as defaults unless an error maps to something else.
However the macros have not been amended since their introduction in 2006 with commit 6c2b374d7485 ("PCI-Express AER implemetation: AER core and aerdriver"). They are still based on PCIe r1.0 sec 7.2.5 (renumbered to 6.2.7 in PCIe r1.1 and newer).
Amend the macros to map errors introduced since then to the appropriate agent and layer.
PCIe r2.1 introduced a new "Component" agent and "General" layer for Internal Errors and Header Log Overflow. Add them to the macros.
Unsupported Request is currently mapped to the "Requester" agent, even though it is reported by the "Receiver". Fix the incorrect mapping.
Sec 6.2.7 neglects to list an agent for Data Link Protocol Error and Surprise Down Error. Map the latter to "Component" because PCIe r7.0 sec 3.2.1 states that the error is "associated with the detecting Port". Map the former to "Receiver" because every occurrence of Data Link Protocol Error in the spec refers to it being logged in the Receiving Port. I have had these errata reported to the PCI-SIG Protocol Working Group. (There's also a layout erratum in the REPLAY_NUM Rollover row wherein columns are shifted to the left, but that's already corrected in the PCIe r7.1 draft as of 2026-04-07.)
--- drivers/pci/pcie/aer.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-)
drivers/pci/pcie/aer.c +19 -5
@@ -428,23 +428,32 @@ void pci_aer_exit(struct pci_dev *dev)#define AER_AGENT_REQUESTER 1#define AER_AGENT_COMPLETER 2#define AER_AGENT_TRANSMITTER 3 +#define AER_AGENT_COMPONENT 4#define AER_AGENT_REQUESTER_MASK (t) ((t == AER_CORRECTABLE) ? \ - 0 : (PCI_ERR_UNC_COMP_TIME|PCI_ERR_UNC_UNSUP)) + 0 : PCI_ERR_UNC_COMP_TIME)#define AER_AGENT_COMPLETER_MASK (t) ((t == AER_CORRECTABLE) ? \ 0 : PCI_ERR_UNC_COMP_ABORT)#define AER_AGENT_TRANSMITTER_MASK (t) ((t == AER_CORRECTABLE) ? \ - (PCI_ERR_COR_REP_ROLL|PCI_ERR_COR_REP_TIMER) : 0) + (PCI_ERR_COR_REP_ROLL|PCI_ERR_COR_REP_TIMER) : \ + (PCI_ERR_UNC_POISON_BLK|PCI_ERR_UNC_ATOMEG| \ + PCI_ERR_UNC_DMWR_BLK|PCI_ERR_UNC_XLAT_BLK| \ + PCI_ERR_UNC_TLPPRE)) +#define AER_AGENT_COMPONENT_MASK (t) ((t == AER_CORRECTABLE) ? \ + (PCI_ERR_COR_INTERNAL|PCI_ERR_COR_LOG_OVER) : \ + (PCI_ERR_UNC_INTN|PCI_ERR_UNC_SURPDN))#define AER_GET_AGENT (t, e) \ ((e & AER_AGENT_COMPLETER_MASK(t)) ? AER_AGENT_COMPLETER : \ (e & AER_AGENT_REQUESTER_MASK(t)) ? AER_AGENT_REQUESTER : \ (e & AER_AGENT_TRANSMITTER_MASK(t)) ? AER_AGENT_TRANSMITTER : \ + (e & AER_AGENT_COMPONENT_MASK(t)) ? AER_AGENT_COMPONENT : \ AER_AGENT_RECEIVER)#define AER_PHYSICAL_LAYER_ERROR 0#define AER_DATA_LINK_LAYER_ERROR 1#define AER_TRANSACTION_LAYER_ERROR 2 +#define AER_GENERAL_ERROR 3#define AER_PHYSICAL_LAYER_ERROR_MASK (t) ((t == AER_CORRECTABLE) ? \ PCI_ERR_COR_RCVR : 0)
@@ -452,11 +461,14 @@ void pci_aer_exit(struct pci_dev *dev) (PCI_ERR_COR_BAD_TLP| \ PCI_ERR_COR_BAD_DLLP| \ PCI_ERR_COR_REP_ROLL| \ - PCI_ERR_COR_REP_TIMER) : PCI_ERR_UNC_DLP) + PCI_ERR_COR_REP_TIMER) : (PCI_ERR_UNC_DLP|PCI_ERR_UNC_SURPDN)) +#define AER_GENERAL_ERROR_MASK (t) ((t == AER_CORRECTABLE) ? \ + (PCI_ERR_COR_INTERNAL|PCI_ERR_COR_LOG_OVER) : PCI_ERR_UNC_INTN)#define AER_GET_LAYER_ERROR (t, e) \ ((e & AER_PHYSICAL_LAYER_ERROR_MASK(t)) ? AER_PHYSICAL_LAYER_ERROR : \ (e & AER_DATA_LINK_LAYER_ERROR_MASK(t)) ? AER_DATA_LINK_LAYER_ERROR : \ + (e & AER_GENERAL_ERROR_MASK(t)) ? AER_GENERAL_ERROR : \ AER_TRANSACTION_LAYER_ERROR)/*
@@ -471,7 +483,8 @@ static const char * const aer_error_severity_string[] = {static const char *aer_error_layer[] = {"Physical Layer" ,"Data Link Layer" , -"Transaction Layer" +"Transaction Layer" , +"General" , };static const char *aer_correctable_error_string[] = {
@@ -548,7 +561,8 @@ static const char *aer_agent_string[] = {"Receiver ID" ,"Requester ID" ,"Completer ID" , -"Transmitter ID" +"Transmitter ID" , + "Component ID", }; #define aer_stats_dev_attr(name, stats_array, strings_array, \
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH 1/6] PCI/AER: Fix mapping of errors to agent & layer
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines
PCIe r7.0 sec 6.2.7 documents the agent and layer of each Correctable and Uncorrectable Error. Based on this spec section, the AER driver maps detected errors to an agent and layer using a set of macros and logs them.
Most errors listed in sec 6.2.7 map to the "Receiver" agent and "Transaction Layer", so the macros use these as defaults unless an error maps to something else.
However the macros have not been amended since their introduction in 2006 with commit
6c2b374d7485("PCI-Express AER implemetation: AER core and aerdriver"). They are still based on PCIe r1.0 sec 7.2.5 (renumbered to 6.2.7 in PCIe r1.1 and newer).
Quoted 8 lines
Amend the macros to map errors introduced since then to the appropriate agent and layer.
[ ... ]
also a layout erratum in theREPLAY_NUMRollover row wherein columns are shifted to the left, but that's already corrected in the PCIe r7.1 draft as of 2026-04-07.)Signed-off-by Lukas Wunner <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
PATCH 2/6 PCI/AER: Log agent & layer for each individual error
In reply to Lukas WunnerThe AER driver maps detected errors to the corresponding agent and layer per PCIe r7.0 sec 6.2.7 and logs both.
If multiple errors were detected, their agent and layer may differ. However the AER driver only logs one agent and one layer for all of them, which seems nonsensical.
Log the agent and layer for each individual error instead.
--- drivers/pci/pcie/aer.c | 42 ++++++++++++++++++++---------------------- 1 file changed, 20 insertions(+), 22 deletions(-)
drivers/pci/pcie/aer.c +20 -22
@@ -558,11 +558,11 @@ static const char *aer_uncorrectable_error_string[] = { };static const char *aer_agent_string[] = { -"Receiver ID" , -"Requester ID" , -"Completer ID" , -"Transmitter ID" , -"Component ID" , +"Receiver" , +"Requester" , +"Completer" , +"Transmitter" , +"Component" , }; #define aer_stats_dev_attr(name, stats_array, strings_array, \
@@ -844,8 +844,8 @@ static void __aer_print_error(struct pci_dev *dev, struct aer_err_info *info) {const char **strings;unsigned long status = info->status & ~info->mask; +const char *errmsg, *agent, *layer;const char *level = info->level; -const char *errmsg;int i;if (info->severity ==AER_CORRECTABLE )
@@ -855,10 +855,17 @@ static void __aer_print_error(struct pci_dev *dev, struct aer_err_info *info)for_each_set_bit (i, &status,32 ) { errmsg = strings[i]; -if (!errmsg) + agent = aer_agent_string[AER_GET_AGENT (info->severity,BIT (i))]; + layer = aer_error_layer[AER_GET_LAYER_ERROR (info->severity, +BIT (i))]; +if (!errmsg) { errmsg ="Unknown Error Bit" ; + agent ="" ; + layer ="" ; + } -aer_printk (level, dev," [%2d] %-22s%s\n" , i, errmsg, +aer_printk (level, dev," [%2d] %-17s | %-11s | %-17s%s\n" , + i, errmsg, agent, layer, info->first_error == i ?" (First)" :"" ); } }
@@ -879,9 +886,9 @@ static void aer_print_source(struct pci_dev *dev, struct aer_err_info *info,void aer_print_error (struct aer_err_info *info,int i) {struct pci_dev *dev; -int layer, agent, id;const char *level = info->level;const char *bus_type =aer_err_bus (info); +int id;if (WARN_ON_ONCE (i >=AER_MAX_MULTI_ERR_DEVICES ))return ;
@@ -897,17 +904,13 @@ void aer_print_error(struct aer_err_info *info, int i)return ;if (!info->status) { -pci_err (dev,"%s Bus Error: severity=%s, type=Inaccessible, (Unregistered Agent ID)\n" , +pci_err (dev,"%s Bus Error: severity=%s (Inaccessible)\n" , bus_type, aer_error_severity_string[info->severity]); goto out; } - layer =AER_GET_LAYER_ERROR (info->severity, info->status); - agent =AER_GET_AGENT (info->severity, info->status); - -aer_printk (level, dev,"%s Bus Error: severity=%s, type=%s, (%s)\n" , - bus_type, aer_error_severity_string[info->severity], - aer_error_layer[layer], aer_agent_string[agent]); +aer_printk (level, dev,"%s Bus Error: severity=%s\n" , + bus_type, aer_error_severity_string[info->severity]);aer_printk (level, dev," device [%04x:%04x] error status/mask=%08x/%08x\n" , dev->vendor, dev->device, info->status, info->mask);
@@ -940,8 +943,8 @@ EXPORT_SYMBOL_GPL(cper_severity_to_aer);void pci_print_aer (struct pci_dev *dev,int aer_severity,struct aer_capability_regs *aer) { +int tlp_header_valid =0 ;const char *bus_type; -int layer, agent, tlp_header_valid =0 ;u32 status, mask;struct aer_err_info info = { .severity = aer_severity,
@@ -972,14 +975,9 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity,if (!aer_ratelimit (dev, info.severity))return ; - layer =AER_GET_LAYER_ERROR (aer_severity, status); - agent =AER_GET_AGENT (aer_severity, status); -aer_printk (info.level, dev,"aer_status: 0x%08x, aer_mask: 0x%08x\n" , status, mask);__aer_print_error (dev, &info); -aer_printk (info.level, dev,"aer_layer=%s, aer_agent=%s\n" , - aer_error_layer[layer], aer_agent_string[agent]);if (aer_severity !=AER_CORRECTABLE ) aer_printk(info.level, dev, "aer_uncor_severity: 0x%08x\n",
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH 2/6] PCI/AER: Log agent & layer for each individual error
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
The AER driver maps detected errors to the corresponding agent and layer per PCIe r7.0 sec 6.2.7 and logs both.
If multiple errors were detected, their agent and layer may differ. However the AER driver only logs one agent and one layer for all of them, which seems nonsensical.
Log the agent and layer for each individual error instead.
Signed-off-by Lukas Wunner <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
PATCH 3/6 PCI/AER: Deduplicate logging of Error Source Identification
In reply to Lukas Wunneraer_print_source() already logs the Error Source Identification Register:
AER: Multiple Correctable error message received from 0000:b7:02.0
However aer_print_error() subsequently identifies the Error Source once more by emitting an "Error of this Agent is reported first" message.
The additional message was introduced by commit 0d465f23502e ("PCI: pcie,
aer: fix report of multiple errors") because it deemed the message emitted by aer_print_source() confusing: When the Multiple ERR_COR Received or Multiple ERR_FATAL/NONFATAL Received bit in the Root Error Status Register is set, it doesn't mean that all errors originated from the device in the Error Source Identification Register. Rather, the errors may have come from multiple distinct devices. The commit sought to make that clearer.
Achieve the commit's objective by rephrasing the message emitted by aer_print_source() and drop the additional message logged by aer_print_error() to reduce dmesg noisiness and simplify the code.
While modifying the log message anyway, fix minor grammatical issues:
Append a plural "s" to "message", add a missing closing brace to "(no details found" and capitalize "Error" to match the spec.
--- drivers/pci/pcie/aer.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-)
drivers/pci/pcie/aer.c +4 -9
@@ -875,12 +875,13 @@ static void aer_print_source(struct pci_dev *dev, struct aer_err_info *info, {u16 source = info->id; -pci_info (dev,"%s%s error message received from %04x:%02x:%02x.%d%s\n" , +pci_info (dev,"%s%s Error message%s from %04x:%02x:%02x.%d%s\n" , info->multi_error_valid ?"Multiple " :"" , aer_error_severity_string[info->severity], + info->multi_error_valid ?"s received, first one" :" received" ,pci_domain_nr (dev->bus),PCI_BUS_NUM (source),PCI_SLOT (source),PCI_FUNC (source), - found ?"" :" (no details found" ); + found ?"" :" (no details found)" ); }void aer_print_error (struct aer_err_info *info,int i)
@@ -888,13 +889,11 @@ void aer_print_error(struct aer_err_info *info, int i)struct pci_dev *dev;const char *level = info->level;const char *bus_type =aer_err_bus (info); -int id;if (WARN_ON_ONCE (i >=AER_MAX_MULTI_ERR_DEVICES ))return ; dev = info->dev[i]; - id =pci_dev_id (dev);pci_dev_aer_stats_incr (dev, info); trace_aer_event(pci_name(dev), (info->status & ~info->mask),
@@ -906,7 +905,7 @@ void aer_print_error(struct aer_err_info *info, int i)if (!info->status) {pci_err (dev,"%s Bus Error: severity=%s (Inaccessible)\n" , bus_type, aer_error_severity_string[info->severity]); - goto out; +return ; } aer_printk(level, dev, "%s Bus Error: severity=%s\n",
@@ -919,10 +918,6 @@ void aer_print_error(struct aer_err_info *info, int i)if (info->tlp_header_valid)pcie_print_tlp_log (dev, &info->tlp, level,dev_fmt (" " )); - -out: -if (info->id && info->error_dev_num >1 && info->id == id) -pci_err (dev," Error of this Agent is reported first\n" ); }#ifdef CONFIG_ACPI_APEI_PCIEAER
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH 3/6] PCI/AER: Deduplicate logging of Error Source Identification
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 19 lines
aer_print_source()already logs the Error Source Identification Register:AER: Multiple Correctable error message received from 0000:b7:02.0
However
aer_print_error()subsequently identifies the Error Source once more by emitting an "Error of this Agent is reported first" message.The additional message was introduced by commit
0d465f23502e("PCI: pcie,
aer: fix report of multiple errors") because it deemed the message emitted byaer_print_source()confusing: When the MultipleERR_CORReceived or Multiple ERR_FATAL/NONFATAL Received bit in the Root Error Status Register is set, it doesn't mean that all errors originated from the device in the Error Source Identification Register. Rather, the errors may have come from multiple distinct devices. The commit sought to make that clearer.Achieve the commit's objective by rephrasing the message emitted by
aer_print_source()and drop the additional message logged byaer_print_error()to reduce dmesg noisiness and simplify the code.
Quoted 5 lines
While modifying the log message anyway, fix minor grammatical issues:
Append a plural "s" to "message", add a missing closing brace to "(no details found" and capitalize "Error" to match the spec.Signed-off-by Lukas Wunner <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
PATCH 4/6 PCI/AER: Emit TLP Log only for unmasked errors
In reply to Lukas WunnerPer PCIe r7.0 sec 6.2.5, the prefix and header of an offending TLP is only recorded for unmasked Uncorrectable Errors. Yet when the AER driver determines whether a prefix and header has been logged, it does not take the Uncorrectable Error Mask Register into account. Fix it.
--- drivers/pci/pcie/aer.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/pci/pcie/aer.c +3 -2
@@ -954,7 +954,8 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity, status = aer->uncor_status; mask = aer->uncor_mask; info.level =KERN_ERR ; - tlp_header_valid =tlp_header_logged (status, aer->cap_control); + tlp_header_valid =tlp_header_logged (status & ~mask, + aer->cap_control); } info.status = status;
@@ -1343,7 +1344,7 @@ int aer_get_device_error_info(struct aer_err_info *info, int i)pci_read_config_dword (dev, aer +PCI_ERR_CAP , &aercc); info->first_error =PCI_ERR_CAP_FEP (aercc); -if (tlp_header_logged (info->status, aercc)) { +if (tlp_header_logged (info->status & ~info->mask, aercc)) { info->tlp_header_valid =1 ; pcie_read_tlp_log(dev, aer + PCI_ERR_HEADER_LOG, aer + PCI_ERR_PREFIX_LOG,
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH 4/6] PCI/AER: Emit TLP Log only for unmasked errors
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 7 lines
Per PCIe r7.0 sec 6.2.5, the prefix and header of an offending TLP is only recorded for unmasked Uncorrectable Errors. Yet when the AER driver determines whether a prefix and header has been logged, it does not take the Uncorrectable Error Mask Register into account. Fix it.
Fixes 6c2b374d7485 ("PCI-Express AER implemetation: AER core and aerdriver")Signed-off-by Lukas Wunner <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
PATCH 5/6 PCI/AER: Move retrieval of FEP and TLP Log into helper
In reply to Lukas WunnerWhen aer_get_device_error_info() gathers information on Uncorrectable Errors from a device, it reads the First Error Pointer and TLP Prefix/ Header Log and caches them in struct aer_err_info.
Those two fields will also need to be read for Advisory Non-Fatal Errors (which are signaled as Correctable Errors). Move their retrieval into a new aer_get_uncor_info() helper for reuse by the imminent Advisory Non-Fatal Error support.
No functional change intended.
--- drivers/pci/pcie/aer.c | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-)
drivers/pci/pcie/aer.c +22 -13
@@ -1289,6 +1289,27 @@ void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn,EXPORT_SYMBOL_GPL (aer_recover_queue);#endif +static void aer_get_uncor_info (struct pci_dev *dev,struct aer_err_info *info, +u32 status) +{ +u16 aer = dev->aer_cap; +u32 aercc; + +/* Get First Error Pointer */ +pci_read_config_dword (dev, aer +PCI_ERR_CAP , &aercc); + info->first_error =PCI_ERR_CAP_FEP (aercc); + +/* Get TLP Prefix/Header Log */ +if (tlp_header_logged (status, aercc)) { + info->tlp_header_valid =1 ; +pcie_read_tlp_log (dev, aer +PCI_ERR_HEADER_LOG , + aer +PCI_ERR_PREFIX_LOG , +aer_tlp_log_len (dev, aercc), + aercc &PCI_ERR_CAP_TLP_LOG_FLIT , + &info->tlp); + } +} +/** * aer_get_device_error_info - read error status from dev and store it to info * @info: pointer to structure to store the error record
@@ -1302,7 +1323,6 @@ int aer_get_device_error_info(struct aer_err_info *info, int i) {struct pci_dev *dev;int type, aer; -u32 aercc;if (i >=AER_MAX_MULTI_ERR_DEVICES )return 0 ;
@@ -1340,18 +1360,7 @@ int aer_get_device_error_info(struct aer_err_info *info, int i)if (!(info->status & ~info->mask))return 0 ; -/* Get First Error Pointer */ -pci_read_config_dword (dev, aer +PCI_ERR_CAP , &aercc); - info->first_error =PCI_ERR_CAP_FEP (aercc); - -if (tlp_header_logged (info->status & ~info->mask, aercc)) { - info->tlp_header_valid =1 ; -pcie_read_tlp_log (dev, aer +PCI_ERR_HEADER_LOG , - aer +PCI_ERR_PREFIX_LOG , -aer_tlp_log_len (dev, aercc), - aercc &PCI_ERR_CAP_TLP_LOG_FLIT , - &info->tlp); - } +aer_get_uncor_info (dev, info, info->status & ~info->mask); }return 1 ;
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH 5/6] PCI/AER: Move retrieval of FEP and TLP Log into helper
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 4 lines
When
aer_get_device_error_info()gathers information on Uncorrectable Errors from a device, it reads the First Error Pointer and TLP Prefix/ Header Log and caches them in structaer_err_info.
Quoted 8 lines
Those two fields will also need to be read for Advisory Non-Fatal Errors (which are signaled as Correctable Errors). Move their retrieval into a new
aer_get_uncor_info()helper for reuse by the imminent Advisory Non-Fatal Error support.No functional change intended.
Signed-off-by Lukas Wunner <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
PATCH 6/6 PCI/AER: Support Advisory Non-Fatal Errors
In reply to Lukas WunnerPer PCIe r7.0 sec 6.2.4.3, certain Non-Fatal Errors may be signaled using ERR_COR instead of ERR_NONFATAL. These "Advisory Non-Fatal Errors" are listed in sec 6.2.7 and explained in detail in sec 6.2.3.2.4.
Advisory Non-Fatal Errors set bits in the Uncorrectable Error Status Register as well as one bit in the Correctable Error Status Register (Advisory Non-Fatal Error Status, bit 13). The latter is masked by default, hence these errors are currently not signaled at all (except on non-compliant products which choose to unmask the bit).
Unmask Advisory Non-Fatal Errors on device enumeration.
Some Non-Fatal Errors are always Advisory, others may be Advisory at the discretion of the detecting agent. If multiple errors occur, the agent may qualify a portion as non-Advisory and signal ERR_NONFATAL in addition to ERR_COR. In this case, there's no way to determine which Non-Fatal Error was Advisory. Assume none is to ensure that the Uncorrectable Error code path is taken to recover from the errors.
Introduce aer_compute_anfe_status() to compute Advisory Non-Fatal Error bits from AER registers, based on this policy. Use it for Firmware First error handling in pci_print_aer(), which receives an AER register dump from the platform (UEFI r2.11 sec N.2.7).
Introduce aer_get_anfe_status() to read AER registers from a device and feed them to aer_compute_anfe_status(). Use it for native error handling in aer_get_device_error_info(), which gathers registers from the device and caches the computed Advisory Non-Fatal Error bits in a new anfe_status field in struct aer_err_info.
Regardless whether error handling is native or Firmware First, the AER driver needs to increment error counters, signal a trace event and log each error. When Advisory Non-Fatal Errors occur, these steps must be performed for Correctable Errors and for Uncorrectable Errors. Achieve this through a recursive invocation of aer_print_error() (for native error handling) and pci_print_aer() (for Firmware First error handling). The recursive invocation reports the (Advisory) Uncorrectable Errors after reporting the Correctable Errors.
Note that the First Error Pointer and TLP Prefix Log is only meaningful for Uncorrectable Errors, but when Advisory Non-Fatal Errors occur, aer_get_device_error_info() has to populate the first_error and tlp_header_valid fields in struct aer_err_info for a Correctable Error. Avoid incorrectly logging those fields for Correctable Errors by amending __aer_print_error() and aer_print_error() with conditionals.
Sample log output for an Advisory Unsupported Request Error:
pcieport 0001:00:00.4: AER: Multiple Correctable Error messages received, first one from 0001:0e:00.0 idxd 0001:0e:00.0: PCIe Bus Error: severity=Correctable
idxd 0001:0e:00.0: device [8086:1216] error status/mask=00002000/00000000
idxd 0001:0e:00.0: [13] NonFatalErr | |
idxd 0001:0e:00.0: PCIe Bus Error: severity=Uncorrectable (Non-Fatal)
idxd 0001:0e:00.0: device [8086:1216] error status/mask=00100000/00000000
idxd 0001:0e:00.0: [20] UnsupReq | Receiver | Transaction Layer (First)
idxd 0001:0e:00.0: AER: TLP Header (Flit): 0x01000104 0x00000000 0x0000080e 0x0f800001
This commit takes inspiration (but differs significantly) from an earlier submission by Zhenzhong Duan, which in turn was based on a submission by Qingshun Wang:
https://lore.kernel.org/r/[email protected]/
Prior attempts at supporting Advisory Non-Fatal Errors were submitted by Yicong Yang and Dio Sun:
https://lore.kernel.org/r/[email protected]/
https://lore.kernel.org/r/BJXPR01MB0614C01A9523786117B1F1CBCEC8A@BJXPR01MB0614.CHNPR01.prod.partner.outlook.cn/
--- drivers/pci/pci.h | 4 ++ drivers/pci/pcie/aer.c | 141 ++++++++++++++++++++++++++++++++++++++--- 2 files changed, 137 insertions(+), 8 deletions(-)
drivers/pci/pci.h +4 -0
@@ -828,6 +828,9 @@ static inline bool pci_dev_binding_disallowed(struct pci_dev *dev)* @tlp_header_valid: Indicates if TLP field contains error information * @status: COR/UNCOR error status * @mask: COR/UNCOR mask +* @anfe_status: Advisory Non-Fatal Errors, i.e. Uncorrectable Errors signaled +* as Correctable Errors (PCIe r7.0 sec 6.2.4.3). Only used if @severity +* is AER_CORRECTABLE and @status has Advisory Non-Fatal Error Status set. * @tlp: Transaction packet information */ struct aer_err_info {
@@ -850,6 +853,7 @@ struct aer_err_info {unsigned int status;unsigned int mask; +u32 anfe_status;struct pcie_tlp_log tlp; };
drivers/pci/pcie/aer.c +133 -8
278 lines not on this page · raw message
Signature · 1 line
2.53.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
New issues:
- [High] The OS unconditionally modifies the AER Correctable Error Mask register, violating the Firmware First architecture.
- [Medium] Native AER incorrectly suppresses logging and uses the wrong log level for Advisory Non-Fatal Errors.
Pre-existing issues:
- [High] PCI device reference leak if hardware error status changes between loops in aer_process_err_devices().
- [High] Hardware errors are silently dropped and permanently lost due to a TOCTOU race condition in aer_process_err_devices().
- [High] Data corruption when decoding concurrent Machine Check Exceptions due to an unprotected global buffer in acpi_extlog.
- [Medium] Memory leak of dynamically allocated aer_capability_regs when the recovery queue overflows.
--
--- Patch [6]: [PATCH 6/6] PCI/AER: Support Advisory Non-Fatal Errors --- commit edbae3f971fbe817c3d83a5348c5dd09f2b62fde
Author: Lukas Wunner <[email protected]>
PCI/AER: Support Advisory Non-Fatal Errors
This commit introduces support for Advisory Non-Fatal Errors (ANFE) by unmasking them during device enumeration. It updates the native and Firmware First error handling paths to compute ANFE status, increment counters, and trace events recursively for both Correctable and (Advisory) Uncorrectable Errors.
Quoted 4 lines
drivers/pci/pcie/aer.c +0
-0
[ ... ]
drivers/pci/pcie/aer.c void pci_aer_init(struct pci_dev *dev) · line 434
Quoted 12 lines
drivers/pci/pcie/aer.c +8
-0@@ -410,6 +424,15 @@ void pci_aer_init(struct pci_dev *dev) n =pcie_cap_has_rtctl (dev) ?5 :4 ;pci_add_ext_cap_save_buffer (dev,PCI_EXT_CAP_ID_ERR ,sizeof (u32) * n); +/* +* Advisory Non-Fatal Errors are masked by default (PCIe r7.0, sec +* 7.8.4.6). +*/ +if (dev->devcap &PCI_EXP_DEVCAP_RBER ) +pci_clear_and_set_config_dword (dev, + dev->aer_cap +PCI_ERR_COR_MASK , +PCI_ERR_COR_ADV_NFAT ,0 );
[Severity: High]
Does this code modify the AER Correctable Error Mask register unconditionally without checking pcie_aer_is_native(dev)?
In systems using Firmware First, the firmware owns the AER hardware and the OS must not modify AER configuration registers. While preexisting functions like pci_aer_clear_status() correctly check pcie_aer_is_native() to respect this boundary, writing to the hardware register unconditionally here could trigger unexpected hardware interrupts to the firmware, potentially causing firmware crashes or system instability.
[ ... ]
Quoted 6 lines
drivers/pci/pcie/aer.c +1
-1@@ -916,8 +997,20 @@ void aer_print_error(struct aer_err_info *info, int i)__aer_print_error (dev, info); -if (info->tlp_header_valid) +if (info->tlp_header_valid && info->severity !=AER_CORRECTABLE )
drivers/pci/pcie/aer.c void aer_print_error(struct aer_err_info *info, int i) · line 1010
Quoted 10 lines
drivers/pci/pcie/aer.c +9
-0pcie_print_tlp_log (dev, &info->tlp, level,dev_fmt (" " )); + +anfe: +/* Recursive invocation for Advisory Non-Fatal Errors */ +if (info->anfe_status && info->severity ==AER_CORRECTABLE ) { + info->severity =AER_NONFATAL ; + info->status = info->anfe_status; + info->mask =0 ; + +aer_print_error (info, i);
[Severity: Medium]
Will this recursive call operate with the stale info->level and info->ratelimit_print[i] state?
Native AER incorrectly suppresses logging and uses the wrong log level for Advisory Non-Fatal Errors. It reuses the pre-computed info->ratelimit_print[i] (evaluated for the parent correctable error) instead of dynamically checking aer_ratelimit() for non-fatal severity. Consequently, if correctable errors are ratelimited, the non-fatal errors are silently dropped without being printed.
Furthermore, it inherits info->level = KERN_WARNING, meaning non-fatal errors are logged at the wrong severity level. Firmware First correctly avoids this bug by dynamically re-evaluating both in pci_print_aer().
[Severity: High]
This is a pre-existing issue, but can this cause a PCI device reference leak if the hardware error status changes between loops?
drivers/pci/pcie/aer.c:add_error_device() {
...
e_info->dev[i] = pci_dev_get(dev);
...
}Device references are originally acquired here. Then, a loop calls handle_error_source(), which contains the only corresponding pci_dev_put(dev):
drivers/pci/pcie/aer.c:aer_process_err_devices() {
...
for (i = 0; i < e_info->error_dev_num && e_info->dev[i]; i++) {
if (aer_get_device_error_info(e_info, i))
handle_error_source(e_info->dev[i], e_info);
}
...
}If the hardware or firmware clears the error status after the first loop but before the second loop reads it, aer_get_device_error_info() returns 0. The second loop will skip the handler, and the pci_dev reference is permanently leaked.
[Severity: High]
This is a pre-existing issue, but does this create a TOCTOU race window for newly arriving hardware errors?
drivers/pci/pcie/aer.c:aer_process_err_devices() {
...
for (i = 0; i < e_info->error_dev_num && e_info->dev[i]; i++) {
if (aer_get_device_error_info(e_info, i))
aer_print_error(e_info, i);
}
for (i = 0; i < e_info->error_dev_num && e_info->dev[i]; i++) {
if (aer_get_device_error_info(e_info, i))
handle_error_source(e_info->dev[i], e_info);
}
...
}This function has two sequential loops: the first prints errors, the second handles and clears them. Both loops call aer_get_device_error_info(), which reads the current hardware error status directly from the device registers.
If the hardware asserts a new error bit between the two loops, the second loop will read it, and pci_aer_handle_error() will write it back to the Write-1-to-Clear (W1C) register. This clears the newly arrived error from hardware before it was ever logged by the first loop. The patch expands this issue by extending it to Advisory Non-Fatal Errors.
[Severity: Medium]
This is a pre-existing issue, but does this code cause a memory leak of the dynamically allocated aer_capability_regs when the recovery queue overflows?
drivers/pci/pcie/aer.c:aer_recover_queue() {
...
else
pr_err("buffer overflow in recovery for %04x:%02x:%02x.%x\n",
domain, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
...
}In the GHES subsystem, memory for struct aer_capability_regs is allocated from ghes_estatus_pool and passed as a pointer to aer_recover_queue(). If the bounded aer_recover_ring kfifo is full, it prints an overflow error and returns without freeing the memory.
The aer_regs pointer is abandoned and the memory is permanently leaked, as it is only freed on the success path inside aer_recover_work_func().
[Severity: High]
This is a pre-existing issue, but can this lead to data corruption when decoding concurrent Machine Check Exceptions due to an unprotected global buffer?
drivers/acpi/acpi_extlog.c:extlog_print() {
...
memcpy(elog_buf, (void *)estatus, ELOG_ENTRY_LEN);
/* clear record status to enable BIOS to update it again */
estatus->block_status = 0;
tmp = (struct acpi_hest_generic_status *)elog_buf;
...
}The ACPI extended log driver allocates a single global buffer elog_buf. The MCE decoder callback unconditionally copies the new MCE event payload into this global buffer without any locks.
Since Machine Check Exceptions can trigger concurrently across different CPUs, concurrent executions of the MCE handlers will simultaneously overwrite elog_buf, corrupting the data structures being parsed.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Quoted 5 lines · On Fri, Jul 24, 2026 at 04:27:55PM +0000, [email protected] wrote:
Quoted 5 lines
+++
b/drivers/pci/pcie/aer.cdrivers/pci/pcie/aer.c +0
-0@@ -410,6 +424,15 @@ void pci_aer_init(struct pci_dev *dev) n =pcie_cap_has_rtctl (dev) ?5 :4 ;pci_add_ext_cap_save_buffer (dev,PCI_EXT_CAP_ID_ERR ,sizeof (u32) * n);
Quoted 12 lines
drivers/pci/pcie/aer.c void pci_aer_init(struct pci_dev *dev) · line 434
Quoted 8 lines
drivers/pci/pcie/aer.c +8
-0+/* +* Advisory Non-Fatal Errors are masked by default (PCIe r7.0, sec +* 7.8.4.6). +*/ +if (dev->devcap &PCI_EXP_DEVCAP_RBER ) +pci_clear_and_set_config_dword (dev, + dev->aer_cap +PCI_ERR_COR_MASK , +PCI_ERR_COR_ADV_NFAT ,0 );[Severity: High]
Does this code modify the AER Correctable Error Mask register unconditionally without checkingpcie_aer_is_native(dev)?
Yes, that's a valid point.
Quoted 4 lines
Quoted 4 lines
drivers/pci/pcie/aer.c +0
-0@@ -916,8 +997,20 @@ void aer_print_error(struct aer_err_info *info, int i)__aer_print_error (dev, info);
Quoted 16 lines
drivers/pci/pcie/aer.c void aer_print_error(struct aer_err_info *info, int i) · line 1010
Quoted 12 lines
drivers/pci/pcie/aer.c +10
-1-if (info->tlp_header_valid) +if (info->tlp_header_valid && info->severity !=AER_CORRECTABLE )pcie_print_tlp_log (dev, &info->tlp, level,dev_fmt (" " )); + +anfe: +/* Recursive invocation for Advisory Non-Fatal Errors */ +if (info->anfe_status && info->severity ==AER_CORRECTABLE ) { + info->severity =AER_NONFATAL ; + info->status = info->anfe_status; + info->mask =0 ; + +aer_print_error (info, i);[Severity: Medium]
Will this recursive call operate with the staleinfo->leveland info->ratelimit_print[i] state?
That's intentional. I want the (Advisory) Uncorrectable Errors to be reported with the same loglevel and ratelimiting as the accompanying Correctable Error.
However while native AER handling does this the way I want it to, I missed that Firmware First does not. It uses the loglevel and ratelimiting of (non-Advisory) Uncorrectable Errors. So that's something I'll have to fix.
Quoted 3 lines
[Severity: High]
This is a pre-existing issue, but can this cause a PCI device reference leak if the hardware error status changes between loops?
[...]
Quoted 4 lines
If the hardware or firmware clears the error status after the first loop but before the second loop reads it,
aer_get_device_error_info()returns 0. The second loop will skip the handler, and thepci_devreference is permanently leaked.
Hallucination. If hardware or firmware clears error bits behind our back, it would be broken.
Quoted 4 lines
[Severity: High]
This is a pre-existing issue, but does this create a TOCTOU race window for newly arriving hardware errors?
Quoted 12 lines
drivers/pci/pcie/aer.c:aer_process_err_devices() {
...
for (i = 0; i < e_info->error_dev_num && e_info->dev[i]; i++) {
if (aer_get_device_error_info(e_info, i))
aer_print_error(e_info, i);
}
for (i = 0; i < e_info->error_dev_num && e_info->dev[i]; i++) {
if (aer_get_device_error_info(e_info, i))
handle_error_source(e_info->dev[i], e_info);
}
...
}It's a valid point and already on my todo list, but not something I want to fix as part of this series. The duplicate invocation of aer_get_device_error_info() is not only susceptible to a TOCTOU race, but also very inefficient.
Quoted 12 lines
[Severity: Medium]
This is a pre-existing issue, but does this code cause a memory leak of the dynamically allocatedaer_capability_regswhen the recovery queue overflows?drivers/pci/pcie/aer.c:aer_recover_queue() { ... else pr_err("buffer overflow in recovery for %04x:%02x:%02x.%x\n", domain, bus, PCI_SLOT(devfn), PCI_FUNC(devfn)); ... }
Quoted 7 lines
In the GHES subsystem, memory for struct
aer_capability_regsis allocated fromghes_estatus_pooland passed as a pointer toaer_recover_queue(). If the boundedaer_recover_ringkfifo is full, it prints an overflow error and returns without freeing the memory.The
aer_regspointer is abandoned and the memory is permanently leaked, as it is only freed on the success path insideaer_recover_work_func().
It's a valid point, I've added that to my todo list.
Quoted 15 lines
[Severity: High]
This is a pre-existing issue, but can this lead to data corruption when decoding concurrent Machine Check Exceptions due to an unprotected global buffer?drivers/acpi/acpi_extlog.c:extlog_print() { ... memcpy(elog_buf, (void *)estatus, ELOG_ENTRY_LEN); /* clear record status to enable BIOS to update it again */ estatus->block_status = 0; tmp = (struct acpi_hest_generic_status *)elog_buf; ... }
Quoted 7 lines
The ACPI extended log driver allocates a single global buffer
elog_buf. The MCE decoder callback unconditionally copies the new MCE event payload into this global buffer without any locks.Since Machine Check Exceptions can trigger concurrently across different CPUs, concurrent executions of the MCE handlers will simultaneously overwrite
elog_buf, corrupting the data structures being parsed.
I'm not familiar with MCE handling, I'll have to ask internally whether that's a valid finding.
Thanks,
Lukas
Bjorn Helgaas
AppliedQuoted 25 lines · Lukas Wunner
PCIe r7.0 sec 6.2.4.3 specifies that certain Non-Fatal Errors may be signaled with
ERR_CORinstead ofERR_NONFATAL. These so-called Advisory Non-Fatal Errors are masked by default and hence not reported by Linux.This series seeks to change that. The meat of it is in patch [6/6]. There have been several previous attempts to support Advisory Non-Fatal Errors which are all linked in the patch.
The preceding patches contain prep work. In particular, they ensure that the detecting agent and layer is reported correctly for each error (patch [1/6] and [2/6]), simplify logging of the error source (patch [3/6]), fix reporting of the TLP Prefix/Header Log (patch [4/6]) and move code around (patch [5/6]).
The patches have been tested internally, but further testing would be appreciated to raise the confidence. Thanks!
Lukas Wunner (6):
PCI/AER: Fix mapping of errors to agent & layer PCI/AER: Log agent & layer for each individual error PCI/AER: Deduplicate logging of Error Source Identification PCI/AER: Emit TLP Log only for unmasked errors PCI/AER: Move retrieval of FEP and TLP Log into helper PCI/AER: Support Advisory Non-Fatal Errors
Quoted 3 lines
drivers/pci/pci.h | 4 + drivers/pci/pcie/aer.c | 250 ++++++++++++++++++++++++++++++++--------- 2 files changed, 200 insertions(+), 54 deletions(-)
Applied to pci/aer with the intention of v7.3, thank you!
I noticed a sashiko comment on 6/6 but haven't had time to look at it yet. I left the series on pci/aer for build testing in the meantime.
Quoted 8 lines · Bjorn Helgaas
Quoted 7 lines · Lukas Wunner
PCI/AER: Fix mapping of errors to agent & layer PCI/AER: Log agent & layer for each individual error PCI/AER: Deduplicate logging of Error Source Identification PCI/AER: Emit TLP Log only for unmasked errors PCI/AER: Move retrieval of FEP and TLP Log into helper PCI/AER: Support Advisory Non-Fatal Errors
Quoted 8 lines
Quoted 3 lines
drivers/pci/pci.h | 4 + drivers/pci/pcie/aer.c | 250 ++++++++++++++++++++++++++++++++--------- 2 files changed, 200 insertions(+), 54 deletions(-)Applied to pci/aer with the intention of v7.3, thank you!
I noticed a sashiko comment on 6/6 but haven't had time to look at it yet. I left the series on pci/aer for build testing in the meantime.
Below is a fixup for the first valid finding reported by sashiko. If you could fold that into the top-most commit on pci/aer, I'd be grateful!
There is one other sashiko finding I need to address:
It complained that in the native case, Advisory Non-Fatal Errors are reported with the same loglevel and ratelimiting as the accompanying Correctable Error. But that's intentional. However in the Firmware First case, I got that wrong in that the loglevel and ratelimiting of non-Advisory Non-Fatal Errors is used. I'll come back with another fixup for that!
My rebuttal of sashiko's findings:
https://lore.kernel.org/r/[email protected]
-- >8 --
drivers/pci/pcie/aer.c +1 -1
@@ -428,7 +428,7 @@ void pci_aer_init(struct pci_dev *dev)* Advisory Non-Fatal Errors are masked by default (PCIe r7.0, sec * 7.8.4.6). */ -if (dev->devcap &PCI_EXP_DEVCAP_RBER ) +if (pcie_aer_is_native (dev) && dev->devcap &PCI_EXP_DEVCAP_RBER )pci_clear_and_set_config_dword (dev, dev->aer_cap +PCI_ERR_COR_MASK ,PCI_ERR_COR_ADV_NFAT ,0 );
Quoted 13 lines · Lukas Wunner
Quoted 12 lines · Bjorn Helgaas
Quoted 10 lines · Lukas Wunner
PCI/AER: Fix mapping of errors to agent & layer PCI/AER: Log agent & layer for each individual error PCI/AER: Deduplicate logging of Error Source Identification PCI/AER: Emit TLP Log only for unmasked errors PCI/AER: Move retrieval of FEP and TLP Log into helper PCI/AER: Support Advisory Non-Fatal Errors
drivers/pci/pci.h | 4 + drivers/pci/pcie/aer.c | 250 ++++++++++++++++++++++++++++++++--------- 2 files changed, 200 insertions(+), 54 deletions(-)
Quoted 8 lines
Quoted 4 lines
Applied to pci/aer with the intention of v7.3, thank you!
I noticed a sashiko comment on 6/6 but haven't had time to look at it yet. I left the series on pci/aer for build testing in the meantime.
Below is a fixup for the first valid finding reported by sashiko. If you could fold that into the top-most commit on pci/aer, I'd be grateful!
Fixup folded into "PCI/AER: Support Advisory Non-Fatal Errors", thanks!
Quoted 15 lines
-- >8 --
drivers/pci/pcie/aer.c +1
-1@@ -428,7 +428,7 @@ void pci_aer_init(struct pci_dev *dev)* Advisory Non-Fatal Errors are masked by default (PCIe r7.0, sec * 7.8.4.6). */ -if (dev->devcap &PCI_EXP_DEVCAP_RBER ) +if (pcie_aer_is_native (dev) && dev->devcap &PCI_EXP_DEVCAP_RBER )pci_clear_and_set_config_dword (dev, dev->aer_cap +PCI_ERR_COR_MASK ,PCI_ERR_COR_ADV_NFAT ,0 );
Quoted 8 lines · Lukas Wunner
There is one other sashiko finding I need to address:
It complained that in the native case, Advisory Non-Fatal Errors are reported with the same loglevel and ratelimiting as the accompanying Correctable Error. But that's intentional. However in the Firmware First case, I got that wrong in that the loglevel and ratelimiting of non-Advisory Non-Fatal Errors is used. I'll come back with another fixup for that!
Below please find a fixup for this finding of sashiko. With that, I believe I have addressed all sashiko findings which are not pre-existing issues.
This fixup is intended to be folded into the top-most commit on pci/aer. If you prefer a proper, separate patch to apply on top of the branch, please let me know. Same if you find anything objectionable in this fixup.
Thanks!
-- >8 --
drivers/pci/pcie/aer.c +21 -6
@@ -1028,13 +1028,15 @@ int cper_severity_to_aer(int cper_severity)EXPORT_SYMBOL_GPL (cper_severity_to_aer);#endif -void pci_print_aer (struct pci_dev *dev,int aer_severity, -struct aer_capability_regs *aer) +static void __pci_print_aer (struct pci_dev *dev,int aer_severity, +struct aer_capability_regs *aer, +bool ratelimit_print,const char * level) {const char *bus_type, *sev;int tlp_header_valid =0 ;u32 status, mask;struct aer_err_info info = { + .level = level, .severity = aer_severity, .first_error =PCI_ERR_CAP_FEP (aer->cap_control), };
@@ -1043,12 +1045,10 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity, status = aer->cor_status; mask = aer->cor_mask; sev ="cor" ; - info.level =KERN_WARNING ; }else { status = aer->uncor_status; mask = aer->uncor_mask; sev ="uncor" ; - info.level =KERN_ERR ; tlp_header_valid =tlp_header_logged (status & ~mask, aer->cap_control); }
@@ -1067,7 +1067,7 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity,* For Advisory Non-Fatal Errors, record statistics and tracing * even if ratelimited */ -if (!aer_ratelimit (dev, info.severity)) +if (!ratelimit_print) goto anfe; aer_printk(info.level, dev,
@@ -1094,10 +1094,25 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity,if (anfe_status) { aer->uncor_status = anfe_status; aer->uncor_mask =0 ; -pci_print_aer (dev,AER_NONFATAL , aer); +__pci_print_aer (dev,AER_NONFATAL , aer, + ratelimit_print, level); } } } + +void pci_print_aer (struct pci_dev *dev,int aer_severity, +struct aer_capability_regs *aer) +{ +/* +* Precalculate ratelimit counter and log level so that Advisory +* Non-Fatal Errors are treated like the accompanying Correctable Error +*/ +bool ratelimit_print =aer_ratelimit (dev, aer_severity); +const char *level = aer_severity ==AER_CORRECTABLE ?KERN_WARNING + :KERN_ERR ; + +__pci_print_aer (dev, aer_severity, aer, ratelimit_print, level); +}EXPORT_SYMBOL_GPL (pci_print_aer);/**
cover PCI/AER: Support Advisory Non-Fatal Errors
There is a part of me that wonders whether all of the complexity in AER handling is really worth it...
Advisory error is one of those insane things in the spec.. It feels like the spec makes it unreasonably complicated for the driver to discriminate. What if we decoupled the message received by the root port from checking & logging the AER status registers? In other words, when the root port receives a message we log the severity we received and whether it was multiple errors.
Then, when we get to the device that sent the message we just always check the CE and the UE status registers? If any status is set that is also unmasked then we log the corresponding TLP Header for that. In addition we log the device status register so users can know what severities were signaled.
We can use the Error Message severity received at the root port to decide whether to walk the pci bus and do the error_detected() stuff. If there are multiple UE status bits set at the reporter & Dev Status register says there was a Non Fatal Error as well a Correctable Error I don't think I care if simply logs everything in UE status as a UE, everything in CE status as CE as long as it also tells me the Dev Status Bits that are set.
Going a little further I would be fine with just always checking both CE/UE status because it seems like it simplifies things a lot & two/three extra config reads/writes is almost a nop if you're already at the device probing it for the other AER things.
Thank you!
Quoted 4 lines · Matthew W Carlis
What if we decoupled the message received by the root port from checking & logging the AER status registers? In other words, when the root port receives a message we log the severity we received and whether it was multiple errors.
We already do exactly that in aer_print_source().
Quoted 5 lines
Then, when we get to the device that sent the message we just always check the CE and the UE status registers? If any status is set that is also unmasked then we log the corresponding TLP Header for that. In addition we log the device status register so users can know what severities were signaled.
That's also already being done (in aer_get_device_error_info() + aer_print_error()) , except we only check the status/mask register corresponding to the severity that the Root Port received.
E.g. if the Root Port received ERR_COR, we only read the Correctable Error Status/Mask registers.
Quoted 2 lines
We can use the Error Message severity received at the root port to decide whether to walk the pci bus and do the
error_detected()stuff.
Same here, we already do that. For ERR_COR, only ->cor_error_detected() is invoked at the reporting device, whereas for ERR_NONFATAL and ERR_FATAL, ->error_detected() and the other callbacks are invoked via pcie_do_recovery().
Quoted 10 lines
If there are multiple UE status bits set at the reporter & Dev Status register says there was a Non Fatal Error as well a Correctable Error I don't think I care if simply logs everything in UE status as a UE, everything in CE status as CE as long as it also tells me the Dev Status Bits that are set.
Going a little further I would be fine with just always checking both CE/UE status because it seems like it simplifies things a lot & two/three extra config reads/writes is almost a nop if you're already at the device probing it for the other AER things.
This is where we differ right now from your proposal: Errors received at the Root Port are queued up in a kfifo and we then empty that kfifo one by one. If there is an Uncorrectable Error behind a Correctable Error in the queue for the same device, we handle the two separately.
Would it make sense to combine them? Maybe, but keep in mind that for Uncorrectable Errors, we may have to perform a Secondary Bus Reset to recover from them, which can affect other devices in the same part of the hierarchy. E.g. if a Switch Upstream Port signals ERR_NONFATAL and then its ->error_detected() callback returns PCI_ERS_RESULT_NEED_RESET, the reset will affect the Switch Upstream Port and everything below. That's very different from how we handle Correctable Errors. For those, the driver of a single device just gets a notification via ->cor_error_detected() and that's it.
There are still many bugs and opportunities for simplification in the AER driver, but refactoring it without breaking things is quite difficult and your proposal would be fairly intrusive I'm afraid.
Thanks,
Lukas