x86, MCE: support memory error recovery for both UCNA and Deferred error in machine_check_poll
Subsystems (2)
PATCH 1/1 x86, MCE: support memory error recovery for both UCNA and Deferred error in machine_check_poll
From: Chen Yucong <[email protected]>
dram_ce_error() stems from Boris's patch set. Thanks!
Uncorrected no action required (UCNA) - is a UCR error that is not signaled via a machine check exception and, instead, is reported to system software as a corrected machine check error. UCNA errors indicate that some data in the system is corrupted, but the data has not been consumed and the processor state is valid and you may continue execution on this processor. UCNA errors require no action from system software to continue execution. Note that UCNA errors are supported by the processor only when IA32_MCG_CAP[24] (MCG_SER_P) is set.
-- Intel SDM Volume 3B
Deferred errors are errors that cannot be corrected by hardware, but do not cause an immediate interruption in program flow, loss of data integrity, or corruption of processor state. These errors indicate that data has been corrupted but not consumed. Hardware writes information to the status and address registers in the corresponding bank that identifies the source of the error if deferred errors are enabled for logging. Deferred errors are not reported via machine check exceptions; they can be seen by polling the MCi_STATUS registers.
-- ADM64 APM Volume 2
Above two items, both UCNA and Deferred errors belong to detected errors, but they can't be corrected by hardware, and this is very similar to Software Recoverable Action Optional (SRAO) errors. Therefore, we can take some actions that have been used for handling SRAO errors to handle UCNA and Deferred errors.
--- arch/x86/include/asm/mce.h | 4 ++++ arch/x86/kernel/cpu/mcheck/mce.c | 39 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+)
arch/x86/include/asm/mce.h +4 -0
@@ -34,6 +34,10 @@#define MCI_STATUS_S (1ULL<<56)/* Signaled machine check */ #define MCI_STATUS_AR (1ULL<<55)/* Action required */ +/* AMD-specific bits */ +#define MCI_STATUS_DEFERRED (1ULL<<44)/* declare an uncorrected error */ +#define MCI_STATUS_POISON (1ULL<<43)/* access poisonous data */ +/* * Note that the full MCACOD field of IA32_MCi_STATUS MSR is * bits 15:0. But bit 12 is the 'F' bit, defined for corrected
arch/x86/kernel/cpu/mcheck/mce.c +39 -0
@@ -575,6 +575,35 @@ static void mce_read_aux(struct mce *m, int i) } } +static bool dram_ce_error (struct mce *m) +{ +struct cpuinfo_x86 *c = &boot_cpu_data; + +if (c->x86_vendor ==X86_VENDOR_AMD ) { +/* ErrCodeExt[20:16] */ +u8 xec = (m->status >>16 ) &0x1f ; + +if (m->status &MCI_STATUS_DEFERRED ) +return (xec ==0x0 || xec ==0x8 ); + }else if (c->x86_vendor ==X86_VENDOR_INTEL ) { +/* +* SDM Volume 3B - 15.9.2 Compound Error Codes (Table 15-9) +* +* Bit 7 of the MCACOD field of IA32_MCi_STATUS is used for +* indicating a memory error. But we can't just blindly check +* bit 7 because if bit 8 is set, then this is a cache error, +* and if bit 11 is set, then it is a bus/ interconnect error +* - and either way bit 7 just gives more detail on what +* cache/bus/interconnect error happened. Note that we can +* ignore bit 12, as it's the "filter" bit. +*/ +if ((m->mcgcap &MCG_SER_P ) && (m->status &MCI_STATUS_UC )) +return (m->status &0xef80 ) ==BIT (7 ); + } + +return false; +} +DEFINE_PER_CPU (unsigned, mce_poll_count);/*
@@ -630,6 +659,16 @@ void machine_check_poll(enum mcp_flags flags, mce_banks_t *b)if (!(flags &MCP_TIMESTAMP )) m.tsc =0 ; + +/* +* In the cases where we don't have a valid address after all, +* do not add it into the ring buffer. +*/ +if (dram_ce_error (&m) && (m.status &MCI_STATUS_ADDRV )) { +mce_ring_add (m.addr >>PAGE_SHIFT ); +mce_schedule_work (); + } +/* * Don't get the IP here because it's unlikely to * have anything to do with the actual error location.
Signature · 10 lines
1.7.10.4 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 99 lines · Chen Yucong
From: Chen Yucong <[email protected]>
dram_ce_error()stems from Boris's patch set. Thanks!Uncorrected no action required (UCNA) - is a UCR error that is not signaled via a machine check exception and, instead, is reported to system software as a corrected machine check error. UCNA errors indicate that some data in the system is corrupted, but the data has not been consumed and the processor state is valid and you may continue execution on this processor. UCNA errors require no action from system software to continue execution. Note that UCNA errors are supported by the processor only when IA32_MCG_CAP[24] (
MCG_SER_P) is set.
-- Intel SDM Volume 3BDeferred errors are errors that cannot be corrected by hardware, but do not cause an immediate interruption in program flow, loss of data integrity, or corruption of processor state. These errors indicate that data has been corrupted but not consumed. Hardware writes information to the status and address registers in the corresponding bank that identifies the source of the error if deferred errors are enabled for logging. Deferred errors are not reported via machine check exceptions; they can be seen by polling the
MCi_STATUSregisters.
-- ADM64 APM Volume 2Above two items, both UCNA and Deferred errors belong to detected errors, but they can't be corrected by hardware, and this is very similar to Software Recoverable Action Optional (SRAO) errors. Therefore, we can take some actions that have been used for handling SRAO errors to handle UCNA and Deferred errors.
Signed-off-by Chen Yucong <[email protected]>--- arch/x86/include/asm/mce.h | 4 ++++ arch/x86/kernel/cpu/mcheck/mce.c | 39 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+)arch/x86/include/asm/mce.h +4
-0@@ -34,6 +34,10 @@#define MCI_STATUS_S (1ULL<<56)/* Signaled machine check */ #define MCI_STATUS_AR (1ULL<<55)/* Action required */ +/* AMD-specific bits */ +#define MCI_STATUS_DEFERRED (1ULL<<44)/* declare an uncorrected error */ +#define MCI_STATUS_POISON (1ULL<<43)/* access poisonous data */ +/* * Note that the full MCACOD field of IA32_MCi_STATUS MSR is * bits 15:0. But bit 12 is the 'F' bit, defined for corrected arch/x86/kernel/cpu/mcheck/mce.c +32
-0@@ -575,6 +575,35 @@ static void mce_read_aux(struct mce *m, int i) } } +static bool dram_ce_error (struct mce *m) +{ +struct cpuinfo_x86 *c = &boot_cpu_data; + +if (c->x86_vendor ==X86_VENDOR_AMD ) { +/* ErrCodeExt[20:16] */ +u8 xec = (m->status >>16 ) &0x1f ; + +if (m->status &MCI_STATUS_DEFERRED ) +return (xec ==0x0 || xec ==0x8 ); + }else if (c->x86_vendor ==X86_VENDOR_INTEL ) { +/* +* SDM Volume 3B - 15.9.2 Compound Error Codes (Table 15-9) +* +* Bit 7 of the MCACOD field of IA32_MCi_STATUS is used for +* indicating a memory error. But we can't just blindly check +* bit 7 because if bit 8 is set, then this is a cache error, +* and if bit 11 is set, then it is a bus/ interconnect error +* - and either way bit 7 just gives more detail on what +* cache/bus/interconnect error happened. Note that we can +* ignore bit 12, as it's the "filter" bit. +*/ +if ((m->mcgcap &MCG_SER_P ) && (m->status &MCI_STATUS_UC )) +return (m->status &0xef80 ) ==BIT (7 ); + } + +return false; +} +DEFINE_PER_CPU (unsigned, mce_poll_count);/* @@ -630,6 +659,16 @@ void machine_check_poll(enum mcp_flags flags, mce_banks_t *b)if (!(flags &MCP_TIMESTAMP )) m.tsc =0 ; + +/* +* In the cases where we don't have a valid address after all,
arch/x86/kernel/cpu/mcheck/mce.c void machine_check_poll(enum mcp_flags flags, mce_banks_t *b) · line 674
Quoted 10 lines
arch/x86/kernel/cpu/mcheck/mce.c +7
-0+* do not add it into the ring buffer. +*/ +if (dram_ce_error (&m) && (m.status &MCI_STATUS_ADDRV )) { +mce_ring_add (m.addr >>PAGE_SHIFT ); +mce_schedule_work (); + } +/* * Don't get the IP here because it's unlikely to * have anything to do with the actual error location.
Hi Boris,
Do you have any comments on this patch?
thx!
cyc
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Borislav Petkov
maintainerQuoted 74 lines · Chen Yucong
From: Chen Yucong <[email protected]>
dram_ce_error()stems from Boris's patch set. Thanks!Uncorrected no action required (UCNA) - is a UCR error that is not signaled via a machine check exception and, instead, is reported to system software as a corrected machine check error. UCNA errors indicate that some data in the system is corrupted, but the data has not been consumed and the processor state is valid and you may continue execution on this processor. UCNA errors require no action from system software to continue execution. Note that UCNA errors are supported by the processor only when IA32_MCG_CAP[24] (
MCG_SER_P) is set.
-- Intel SDM Volume 3BDeferred errors are errors that cannot be corrected by hardware, but do not cause an immediate interruption in program flow, loss of data integrity, or corruption of processor state. These errors indicate that data has been corrupted but not consumed. Hardware writes information to the status and address registers in the corresponding bank that identifies the source of the error if deferred errors are enabled for logging. Deferred errors are not reported via machine check exceptions; they can be seen by polling the
MCi_STATUSregisters.
-- ADM64 APM Volume 2Above two items, both UCNA and Deferred errors belong to detected errors, but they can't be corrected by hardware, and this is very similar to Software Recoverable Action Optional (SRAO) errors. Therefore, we can take some actions that have been used for handling SRAO errors to handle UCNA and Deferred errors.
Signed-off-by Chen Yucong <[email protected]>--- arch/x86/include/asm/mce.h | 4 ++++ arch/x86/kernel/cpu/mcheck/mce.c | 39 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+)arch/x86/include/asm/mce.h +4
-0@@ -34,6 +34,10 @@#define MCI_STATUS_S (1ULL<<56)/* Signaled machine check */ #define MCI_STATUS_AR (1ULL<<55)/* Action required */ +/* AMD-specific bits */ +#define MCI_STATUS_DEFERRED (1ULL<<44)/* declare an uncorrected error */ +#define MCI_STATUS_POISON (1ULL<<43)/* access poisonous data */ +/* * Note that the full MCACOD field of IA32_MCi_STATUS MSR is * bits 15:0. But bit 12 is the 'F' bit, defined for corrected arch/x86/kernel/cpu/mcheck/mce.c +14
-0@@ -575,6 +575,35 @@ static void mce_read_aux(struct mce *m, int i) } } +static bool dram_ce_error (struct mce *m) +{ +struct cpuinfo_x86 *c = &boot_cpu_data; + +if (c->x86_vendor ==X86_VENDOR_AMD ) { +/* ErrCodeExt[20:16] */ +u8 xec = (m->status >>16 ) &0x1f ; + +if (m->status &MCI_STATUS_DEFERRED ) +return (xec ==0x0 || xec ==0x8 ); + }else if (c->x86_vendor ==X86_VENDOR_INTEL ) { +/* +* SDM Volume 3B - 15.9.2 Compound Error Codes (Table 15-9) +*
arch/x86/kernel/cpu/mcheck/mce.c static void mce_read_aux(struct mce *m, int i) · line 601
Quoted 10 lines
arch/x86/kernel/cpu/mcheck/mce.c +10
-0+* Bit 7 of the MCACOD field of IA32_MCi_STATUS is used for +* indicating a memory error. But we can't just blindly check +* bit 7 because if bit 8 is set, then this is a cache error, +* and if bit 11 is set, then it is a bus/ interconnect error +* - and either way bit 7 just gives more detail on what +* cache/bus/interconnect error happened. Note that we can +* ignore bit 12, as it's the "filter" bit. +*/ +if ((m->mcgcap &MCG_SER_P ) && (m->status &MCI_STATUS_UC )) +return (m->status &0xef80 ) ==BIT (7 );
This is wrong - dram_ce_error is supposed to tell us whether this is a correctable error, not uncorrectable.
Quoted 11 lines
arch/x86/kernel/cpu/mcheck/mce.c +5
-0+ } + +return false; +} +DEFINE_PER_CPU (unsigned, mce_poll_count);/* @@ -630,6 +659,16 @@ void machine_check_poll(enum mcp_flags flags, mce_banks_t *b)if (!(flags &MCP_TIMESTAMP ))
arch/x86/kernel/cpu/mcheck/mce.c void machine_check_poll(enum mcp_flags flags, mce_banks_t *b) · line 670
Quoted 10 lines
arch/x86/kernel/cpu/mcheck/mce.c +9
-0m.tsc =0 ; + +/* +* In the cases where we don't have a valid address after all, +* do not add it into the ring buffer. +*/ +if (dram_ce_error (&m) && (m.status &MCI_STATUS_ADDRV )) { +mce_ring_add (m.addr >>PAGE_SHIFT ); +mce_schedule_work (); + }
The general idea of preemptively poisoning pages which contain deferred errors is fine though.
You need to figure out how exactly you're going to detect such errors. On AMD it is easy - MCI_STATUS_DEFERRED. The problem there is to find out which models actually set that bit and what the other bits in MCi_STATUS are set to. For that we should ask Aravind.
On Intel, SDM says something like this:
"A UNCA error is indicated with UC=1, PCC=0, S=0 and AR=0 in the IA32_MCi_STATUS register."
I think a proper solution would be to extend the mce_severity mechanism to handle that and return AO_SEVERITY.
What I would love even more, though, if in conjunction with this someone (you?) would sit down and actually make that severities[] array readable because it makes my eyes bleed and my brain twist everytime I look at it.
Thanks.
Signature · 11 lines
Regards/Gruss,
Boris.
Sent from a fat crate under my desk. Formatting is fine.
--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>PiBUaGUgZ2VuZXJhbCBpZGVhIG9mIHByZWVtcHRpdmVseSBwb2lzb25pbmcgcGFnZXMgd2hpY2gg Y29udGFpbiBkZWZlcnJlZA0KPiBlcnJvcnMgaXMgZmluZSB0aG91Z2guDQoNCkFncmVlZC4gSSB1 c2VkIHRvIHRoaW5rIHRoYXQgaXQgd2Fzbid0IGxpa2VseSB0byBiZSB2ZXJ5IHVzZWZ1bCBiZWNh dXNlIGluIG1hbnkNCmNhc2VzIHRoZSBVQ05BIGVycm9ycyBhcmUganVzdCBhIHRyYWlsIG9mIGJy ZWFkY3J1bWJzIHNldCBieSBkaWZmZXJlbnQgdW5pdHMNCm9uIHRoZSBjaGlwIGFzIHRoZSBwb2lz b24gcGFzc2VkIHRocm91Z2ggb24gdGhlIHdheSB0byBjb25zdW1wdGlvbiAtIHdoZXJlDQp0aGVy ZSB3b3VsZCBiZSBhIGZhdGFsIChvciByZWNvdmVyYWJsZSkgZXJyb3IuDQoNCkJ1dCByZWNlbnRs eSBJIGZvdW5kIHRoYXQgYSBwYXJ0aWFsIHdyaXRlIHRvIGEgcG9pc29uZWQgY2FjaGUgbGluZSBv bmx5IHNldHMgdGhlDQp0cmFpbCBvZiBVQ05BIGVycm9ycyAtIHRoZXJlIGlzIG5vIGNvbnN1bXB0 aW9uLCBzbyBubyBtYWNoaW5lIGNoZWNrLiAgU28gaW4NCnRoaXMgY2FzZSBpdCB3b3VsZCBkZWZp bml0ZWx5IGJlIHdvcnRod2hpbGUgdG8gdHJpZ2dlciB0aGUgc2FtZSBhY3Rpb24gdGhhdCB3ZQ0K ZG8gZm9yIFNSQU8gdG8gdW5tYXAgdGhlIHBhZ2UgYmVmb3JlIHNvbWVvbmUgZG9lcyBkbyBhIHJl YWQuDQoNCi1Ub255DQo=
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>