x86/mce: Set PG_hwpoison page flag to avoid the capture kernel panic
Subsystems (3)
Tony Luck
maintainerPATCH 1/1 x86/mce: Set PG_hwpoison page flag to avoid the capture kernel panic
From: Zhiquan Li <[email protected]>
Kdump can exclude the HWPosion page to avoid touch the error page again, the prerequisite is the PG_hwpoison page flag is set. However, for some MCE fatal error cases, there are no opportunity to queue a task for calling memory_failure(), as a result, the capture kernel touches the error page again and panics.
Add function mce_set_page_hwpoison_now() which mark a page as HWPoison before kernel panic() for MCE error, so that the dump program can check and skip the error page and prevent the capture kernel panic.
[Tony: Changed TestSetPageHWPoison() to SetPageHWPoison()]
--- arch/x86/kernel/cpu/mce/core.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
arch/x86/kernel/cpu/mce/core.c +20 -0
@@ -162,6 +162,24 @@ void mce_unregister_decode_chain(struct notifier_block *nb) }EXPORT_SYMBOL_GPL (mce_unregister_decode_chain); +/* +* Kdump can exclude the HWPosion page to avoid touch the error page again, +* the prerequisite is the PG_hwpoison page flag is set. However, for some +* MCE fatal error cases, there are no opportunity to queue a task +* for calling memory_failure(), as a result, the capture kernel panic. +* This function mark the page as HWPoison before kernel panic() for MCE. +*/ +static void mce_set_page_hwpoison_now (unsigned long pfn) +{ +struct page *p; + +/* TODO: need to handle other sort of page, like SGX, PMEM and +* HugeTLB pages*/ + p =pfn_to_online_page (pfn); +if (p) +SetPageHWPoison (p); +} +static void __print_mce (struct mce *m) { pr_emerg(HW_ERR "CPU %d: Machine Check%s: %Lx Bank %d: %016Lx\n",
@@ -292,6 +310,8 @@ static noinstr void mce_panic(const char *msg, struct mce *final, char *exp)if (!fake_panic) {if (panic_timeout ==0 ) panic_timeout = mca_cfg.panic_timeout; +if (final && (final->status &MCI_STATUS_ADDRV )) +mce_set_page_hwpoison_now (final->addr >>PAGE_SHIFT );panic (msg); }else pr_emerg (HW_ERR "Fake kernel panic: %s\n" , msg);
Signature · 1 line
2.39.1
Quoted 13 lines · Tony Luck
From: Zhiquan Li <[email protected]>
Kdump can exclude the HWPosion page to avoid touch the error page again, the prerequisite is the
PG_hwpoisonpage flag is set. However, for some MCE fatal error cases, there are no opportunity to queue a task for callingmemory_failure(), as a result, the capture kernel touches the error page again and panics.Add function
mce_set_page_hwpoison_now()which mark a page as HWPoison before kernelpanic()for MCE error, so that the dump program can check and skip the error page and prevent the capture kernel panic.
Quoted 6 lines
[Tony: Changed
TestSetPageHWPoison()toSetPageHWPoison()]Co-developedd-by Youquan Song <[email protected]>Signed-off-by Youquan Song <[email protected]>Signed-off-by Zhiquan Li <[email protected]>Signed-off-by Tony Luck <[email protected]>
Hi,
Thank you for the patch.
Quoted 9 lines
--- arch/x86/kernel/cpu/mce/core.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)arch/x86/kernel/cpu/mce/core.c +0
-0
arch/x86/kernel/cpu/mce/core.c void mce_unregister_decode_chain(struct notifier_block *nb) · line 169
Quoted 8 lines
arch/x86/kernel/cpu/mce/core.c +5
-0@@ -162,6 +162,24 @@ void mce_unregister_decode_chain(struct notifier_block *nb) }EXPORT_SYMBOL_GPL (mce_unregister_decode_chain); +/* +* Kdump can exclude the HWPosion page to avoid touch the error page again, +* the prerequisite is the PG_hwpoison page flag is set. However, for some +* MCE fatal error cases, there are no opportunity to queue a task +* for calling memory_failure(), as a result, the capture kernel panic.
s/panic/panics/ ?
arch/x86/kernel/cpu/mce/core.c void mce_unregister_decode_chain(struct notifier_block *nb) · line 170
Quoted 1 line
arch/x86/kernel/cpu/mce/core.c +1
-0+* This function mark the page as HWPoison before kernel panic() for MCE.
s/mark/marks/
arch/x86/kernel/cpu/mce/core.c void mce_unregister_decode_chain(struct notifier_block *nb) · line 177
Quoted 7 lines
arch/x86/kernel/cpu/mce/core.c +7
-0+*/ +static void mce_set_page_hwpoison_now (unsigned long pfn) +{ +struct page *p; + +/* TODO: need to handle other sort of page, like SGX, PMEM and +* HugeTLB pages*/
Although I'm not sure that SGX memory or PMEM pages are expected to be included in kdump, but simply setting PageHWPoison does not work for them? (Maybe that depends on how kdump handles these types of memory.)
As for HugeTLB, kdump utility should parse the struct page and be aware of HugeTLB pages, so maybe setting PageHWPoison on the head page could work.
Quoted 9 lines
arch/x86/kernel/cpu/mce/core.c +5
-0+ p =pfn_to_online_page (pfn); +if (p) +SetPageHWPoison (p); +} +static void __print_mce (struct mce *m) { pr_emerg(HW_ERR "CPU %d: Machine Check%s: %Lx Bank %d: %016Lx\n",
arch/x86/kernel/cpu/mce/core.c static noinstr void mce_panic(const char *msg, struct mce *final, char *exp) · line 315
Quoted 6 lines
arch/x86/kernel/cpu/mce/core.c +2
-0@@ -292,6 +310,8 @@ static noinstr void mce_panic(const char *msg, struct mce *final, char *exp)if (!fake_panic) {if (panic_timeout ==0 ) panic_timeout = mca_cfg.panic_timeout; +if (final && (final->status &MCI_STATUS_ADDRV )) +mce_set_page_hwpoison_now (final->addr >>PAGE_SHIFT );panic (msg);
I think that setting PageHWPoison outside hwpoison subsystem is OK here, because this is called just before calling panic() so it's expected to not conflict with other hwpoison-related code.
Thanks,
Naoya Horiguchi
Luck, Tony
maintainerQuoted 3 lines
Although I'm not sure that SGX memory or PMEM pages are expected to be included in kdump, but simply setting PageHWPoison does not work for them? (Maybe that depends on how kdump handles these types of memory.)
SGX/TDX pages can't be dumped. They are encrypted with no way for kdump to get the key.
PMEM seems pointless (but I don't know what kdump does here).
Quoted 2 lines
As for HugeTLB, kdump utility should parse the struct page and be aware of HugeTLB pages, so maybe setting PageHWPoison on the head page could work.
Or maybe kdump can take not of the PageHWPoison flag on the sub-page of the huge page? It depends on whether there is any benefit to the dump to include the not-poisoned parts of a huge page.
Quoted 3 lines
I think that setting PageHWPoison outside hwpoison subsystem is OK here, because this is called just before calling
panic()so it's expected to not conflict with other hwpoison-related code.
Thanks for the review.
-Tony
Quoted 7 lines · Tony
Quoted 3 lines
Although I'm not sure that SGX memory or PMEM pages are expected to be included in kdump, but simply setting PageHWPoison does not work for them? (Maybe that depends on how kdump handles these types of memory.)
SGX/TDX pages can't be dumped. They are encrypted with no way for kdump to get the key.
Quoted 8 lines
PMEM seems pointless (but I don't know what kdump does here).
Quoted 2 lines
As for HugeTLB, kdump utility should parse the struct page and be aware of HugeTLB pages, so maybe setting PageHWPoison on the head page could work.
Or maybe kdump can take not of the PageHWPoison flag on the sub-page of the huge page? It depends on whether there is any benefit to the dump to include the not-poisoned parts of a huge page.
I think that many kdump users filter out HugeTLB pages (setting dump_level to filter "User pages") to reduce the size of kdump. User pages are not much helpful to investigate kernel problems, so filtering all sub-pages in hwpoisoned hugepage seems to me not so harmful.
I don't say that saving healthy subpages has no benefit, but I don't know much about usecases where user pages in kdump file help.
Thanks,
Naoya Horiguchi
Quoted 17 lines · HORIGUCHI NAOYA(堀口 直也)
Quoted 7 lines · Tony
Quoted 3 lines
Although I'm not sure that SGX memory or PMEM pages are expected to be included in kdump, but simply setting PageHWPoison does not work for them? (Maybe that depends on how kdump handles these types of memory.)
SGX/TDX pages can't be dumped. They are encrypted with no way for kdump to get the key.
Quoted 8 lines
PMEM seems pointless (but I don't know what kdump does here).
Quoted 2 lines
As for HugeTLB, kdump utility should parse the struct page and be aware of HugeTLB pages, so maybe setting PageHWPoison on the head page could work.
Or maybe kdump can take not of the PageHWPoison flag on the sub-page of the huge page? It depends on whether there is any benefit to the dump to include the not-poisoned parts of a huge page.
Quoted 8 lines
I think that many kdump users filter out HugeTLB pages (setting
dump_levelto filter "User pages") to reduce the size of kdump. User pages are not much helpful to investigate kernel problems, so filtering all sub-pages in hwpoisoned hugepage seems to me not so harmful.I don't say that saving healthy subpages has no benefit, but I don't know much about usecases where user pages in kdump file help.
Many thanks, Naoya and Tony.
The "TODO" comment was initially added by me. I referenced memory_failure() that gets rid of the three cases specifically, so I'd like the cases are fully discussed here to make sure it is a strong fix.
- SGX, as Tony said those pages can't be dumped.
- PMEM, although the memory region is figured out by kexec, kdump utility doesn't have corresponding implementation for it. So we can ignore it.
- HugeTLB
As we known there are complicated processes for HugeTLB pages atmemory_failure(). However, it doesn't make sense for a routine going to call panic(). Kernel marks the page hwpoisoned would be enough.
The information is sufficient for kdump to make decision on how to deal with a hwpoisoned huge pages, even though setting the PageHWPoison flag on the sub-page of a huge page. It's kdump's responsibility for the not-poisoned parts of a huge page. Currently the implementation is to exclude the whole huge page simply:
-
https://github.com/makedumpfile/makedumpfile/commit/e8b4f93b3260defe86f5e13ca7536c07f2e32914
- https://bugzilla.redhat.com/show_bug.cgi?id=1181649
Best Regards,
Zhiquan
Tony Luck
maintainerPATCH v2 1/1 x86/mce: Set PG_hwpoison page flag to avoid the capture kernel panic
In reply to Tony LuckFrom: Zhiquan Li <[email protected]>
Kdump can exclude the HWPosion page to avoid touch the error page again, the prerequisite is the PG_hwpoison page flag is set. However, for some MCE fatal error cases, there are no opportunity to queue a task for calling memory_failure(), as a result, the capture kernel touches the error page again and panics.
Add function mce_set_page_hwpoison_now() which mark a page as HWPoison before kernel panic() for MCE error, so that the dump program can check and skip the error page and prevent the capture kernel panic.
[Tony: Changed TestSetPageHWPoison() to SetPageHWPoison()]
---
v2: Replaced "TODO" comment in code with comments based on mailing list discussion on the lack of value in covering other page types
arch/x86/kernel/cpu/mce/core.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
arch/x86/kernel/cpu/mce/core.c +26 -0
@@ -156,6 +156,30 @@ void mce_unregister_decode_chain(struct notifier_block *nb) }EXPORT_SYMBOL_GPL (mce_unregister_decode_chain); +/* +* Kdump can exclude the HWPosion page to avoid touch the error page again, +* the prerequisite is the PG_hwpoison page flag is set. However, for some +* MCE fatal error cases, there are no opportunity to queue a task +* for calling memory_failure(), as a result, the capture kernel panic. +* This function mark the page as HWPoison before kernel panic() for MCE. +* +* This covers normal 4KByte pages. There is little/no value in covering +* other page types. E.g. +* SGX: These cannot be dumped. +* PMEM: Pointless to dump these. Persistent memory contents remain +* available across reboots. +* HugeTLB: These are user pages. Generally filtered out of the kdump +* to keep size small. Not helpful to debug kernel issues. +*/ +static void mce_set_page_hwpoison_now (unsigned long pfn) +{ +struct page *p; + + p =pfn_to_online_page (pfn); +if (p) +SetPageHWPoison (p); +} +static void __print_mce (struct mce *m) { pr_emerg(HW_ERR "CPU %d: Machine Check%s: %Lx Bank %d: %016Lx\n",
@@ -286,6 +310,8 @@ static noinstr void mce_panic(const char *msg, struct mce *final, char *exp)if (!fake_panic) {if (panic_timeout ==0 ) panic_timeout = mca_cfg.panic_timeout; +if (final && (final->status &MCI_STATUS_ADDRV )) +mce_set_page_hwpoison_now (final->addr >>PAGE_SHIFT );panic (msg); }else pr_emerg (HW_ERR "Fake kernel panic: %s\n" , msg);
Signature · 1 line
2.40.1