x86/mce: Initialize "bank" when we find a fatal error in mce_no_way_out()
Subsystems (2)
Tony Luck
maintainerPATCH 1/1 x86/mce: Initialize "bank" when we find a fatal error in mce_no_way_out()
Internal injection testing crashed with a console log that said:
mce: [Hardware Error]: CPU 7: Machine Check Exception: f Bank 0: bd80000000100134
This caused a lot of head scratching because the MCACOD (bits 15:0) of that status is a signature from an L1 data cache error. But Linux says that it found it in "Bank 0", which on this model CPU only reports L1 instruction cache errors.
The answer was that Linux doesn't initialize "m->bank" in the case that it finds a fatal error in the mce_no_way_out() pre-scan of banks. If this was a local machine check, then we pass this partially initialized "struct mce" to mce_panic().
Fix is simple. Just initialize m->bank in the case that we found a fatal error.
--- arch/x86/kernel/cpu/mce/core.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kernel/cpu/mce/core.c +1 -0
@@ -784,6 +784,7 @@ static int mce_no_way_out(struct mce *m, char **msg, unsigned long *validp,quirk_no_way_out (i, m, regs);if (mce_severity (m, mca_cfg.tolerant, &tmp, true) >=MCE_PANIC_SEVERITY ) { + m->bank = i;mce_read_aux (m, i); *msg = tmp;return 1 ;
Signature · 1 line
2.19.1
Borislav Petkov
maintainerQuoted 26 lines · Tony Luck
Internal injection testing crashed with a console log that said:
mce: [Hardware Error]: CPU 7: Machine Check Exception: f Bank 0:
bd80000000100134This caused a lot of head scratching because the MCACOD (bits 15:0) of that status is a signature from an L1 data cache error. But Linux says that it found it in "Bank 0", which on this model CPU only reports L1 instruction cache errors.
The answer was that Linux doesn't initialize "
m->bank" in the case that it finds a fatal error in themce_no_way_out()pre-scan of banks. If this was a local machine check, then we pass this partially initialized "struct mce" tomce_panic().Fix is simple. Just initialize
m->bankin the case that we found a fatal error.Fixes 40c36e2741d7 ("x86/mce: Fix incorrect "Machine check from unknown source" message")Cc [email protected] # v4.18 Note pre-v5.0 arch/x86/kernel/cpu/mce/core.c was called arch/x86/kernel/cpu/mcheck/mce.cSigned-off-by Tony Luck <[email protected]>--- arch/x86/kernel/cpu/mce/core.c | 1 + 1 file changed, 1 insertion(+)arch/x86/kernel/cpu/mce/core.c +0
-0
arch/x86/kernel/cpu/mce/core.c static int mce_no_way_out(struct mce *m, char **msg, unsigned long *validp, · line 787
Quoted 4 lines
arch/x86/kernel/cpu/mce/core.c +1
-0@@ -784,6 +784,7 @@ static int mce_no_way_out(struct mce *m, char **msg, unsigned long *validp,quirk_no_way_out (i, m, regs);if (mce_severity (m, mca_cfg.tolerant, &tmp, true) >=MCE_PANIC_SEVERITY ) { + m->bank = i;
So conceptually this write belongs in...
arch/x86/kernel/cpu/mce/core.c static int mce_no_way_out(struct mce *m, char **msg, unsigned long *validp, · line 788
Quoted 1 line
arch/x86/kernel/cpu/mce/core.c +0
-0mce_read_aux (m, i);
... this function, i.e., in mce_read_aux() because it gets the bank number passed in already. And our calling pattern when populating struct mce is:
mce_gather_info() mce_read_aux()
so it'll be more robust if we moved it there.
Also, that argument "i" of mce_read_aux() is not very telling and it should be "bank" but that would complicate the stable backporting so if you feel like it, you could do a second, cleanup patch ontop to fix that too.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.Luck, Tony
maintainerQuoted 7 lines · Borislav Petkov
arch/x86/kernel/cpu/mce/core.c static int mce_no_way_out(struct mce *m, char **msg, unsigned long *validp, · line 787
Quoted 2 lines · Tony Luck
arch/x86/kernel/cpu/mce/core.c +1
-0if (mce_severity (m, mca_cfg.tolerant, &tmp, true) >=MCE_PANIC_SEVERITY ) { + m->bank = i;So conceptually this write belongs in...
Quoted 1 line
mce_read_aux(m, i);
Really? That function starts with the comment
/* * Read ADDR and MISC registers. */
and the name is pretty descriptive that it is "reading auxiliary" information.
Quoted 7 lines
... this function, i.e., in
mce_read_aux()because it gets the bank number passed in already. And our calling pattern when populating struct mce is:mce_gather_info() mce_read_aux()
Only two exising uses:
1) in machine_check_poll()
mce_gather_info(&m, NULL);
for (each bank) {
m.bank = i;
mce_read_aux(&m, i);
...
}2) in __mc_scan_banks
// mce_gather_info() already done in do_machine_check
for (each bank) {
m.bank = i;
mce_read_aux(m, i);
...
}Quoted 2 lines
so it'll be more robust if we moved it there.
It would be redundant to move it there for both
existing uses.
Quoted 4 lines
Also, that argument "i" of
mce_read_aux()is not very telling and it should be "bank" but that would complicate the stable backporting so if you feel like it, you could do a second, cleanup patch ontop to fix that too.
Agreed that "bank" is clearer than "i". But also agreed that should be a separate patch for another day.
-Tony
Borislav Petkov
maintainerQuoted 4 lines · Tony
Quoted 1 line
so it'll be more robust if we moved it there.
It would be redundant to move it there for both
existing uses.
Maybe. But if the bank write happens there, it won't be "forgotten" again.
I don't care what the functions are called. If they need to do something more, like *fully* populating struct mce so that a proper record gets logged further down the line, then we need to make them do so. That's the point I'm trying to make.
Sure, the stable fix should be simple for easier backporting but in order to avoid this thing happening again in the future, we should probably look at unifying and making those paths easier to use.
Right now we have
1. mce_setup() - initial population of struct mce, called from mce_gather_info(), a.o.
2. mce_gather_info() collects global regs
3. mce_read_aux() reading aux regs
and at the very end, we hand it into mce_log().
And in-between we poke (or don't poke) in some fields.
IOW, if the struct mce finishing population is concentrated in a single function, it will be much harder to forget setting such fields in the future.
In this case, the devil is in the detail so I'll have a look at this when more time.
Signature · 4 lines
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.tip-bot for Tony Luck
merge bot[tip:x86/urgent] x86/MCE: Initialize mce.bank in the case of a fatal error in mce_no_way_out()
In reply to Tony Luckx86/MCE: Initialize mce.bank in the case of a fatal error in mce_no_way_out()
Commit-ID: d28af26faa0b1daf3c692603d46bc4687c16f19e Gitweb: https://git.kernel.org/tip/d28af26faa0b1daf3c692603d46bc4687c16f19e Author: Tony Luck <[email protected]> AuthorDate: Thu, 31 Jan 2019 16:33:41 -0800 Committer: Borislav Petkov <[email protected]> CommitDate: Sun, 3 Feb 2019 13:24:24 +0100
x86/MCE: Initialize mce.bank in the case of a fatal error in mce_no_way_out()
Internal injection testing crashed with a console log that said:
mce: [Hardware Error]: CPU 7: Machine Check Exception: f Bank 0: bd80000000100134
This caused a lot of head scratching because the MCACOD (bits 15:0) of that status is a signature from an L1 data cache error. But Linux says that it found it in "Bank 0", which on this model CPU only reports L1 instruction cache errors.
The answer was that Linux doesn't initialize "m->bank" in the case that it finds a fatal error in the mce_no_way_out() pre-scan of banks. If this was a local machine check, then this partially initialized struct mce is being passed to mce_panic().
Fix is simple: just initialize m->bank in the case of a fatal error.
--- arch/x86/kernel/cpu/mce/core.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kernel/cpu/mce/core.c +1 -0
@@ -784,6 +784,7 @@ static int mce_no_way_out(struct mce *m, char **msg, unsigned long *validp,quirk_no_way_out (i, m, regs);if (mce_severity (m, mca_cfg.tolerant, &tmp, true) >=MCE_PANIC_SEVERITY ) { + m->bank = i;mce_read_aux (m, i); *msg = tmp;return 1 ;