Skip to contentSkip to conversation

RFC: Implement hwpoison on free for soft offlining

PATCH In review Andi Kleen 4 participants 10 messages Oct 6, 2010 – Oct 15, 2010

Subsystems (4)
lore
Share & tools
lore

Andi Kleen

author
#0 lore raw

Here's a somewhat experimental patch to improve soft offlining in hwpoison, but allowing hwpoison on free for not directly freeable page types. It should work for nearly all left over page types that get eventually freed, so this makes soft offlining nearly universal. The only non handleable page types are now pages that never get freed.

Drawback: It needs an additional page flag. Cannot set hwpoison directly because that would not be "soft" and cause errors.

Since the flags are scarce on 32bit I only enabled it on 64bit.

Comments?

-Andi

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#1 lore raw

PATCH 1/1 HWPOISON: Implement hwpoison-on-free for soft offlining

From: Andi Kleen <[email protected]>

Hard offlining was always able to offline most kernel pages by setting the HWPoison flag on it and waiting for the next free. This works on all pages that get eventually freed.

This didn't work for soft offlining unfortunately, because it cannot safely set the HWPoison flag on a still alive page. Handle this by introducing a second page flag HWPoisonOnFree that is handled by page free and enables HWPoison then. This does not add any overhead to page free because it can be handled as a bad flag which are already checked for.

Since page flags are scarce on 32bit architectures this is only done on 64bit .

---
 include/linux/mm.h         |    1 +
 include/linux/page-flags.h |   15 +++++++++++++-
 mm/Kconfig                 |    4 +++
 mm/memory-failure.c        |   46 +++++++++++++++++++++++++++++++++++++++++++-
 mm/page_alloc.c            |    2 +
 5 files changed, 66 insertions(+), 2 deletions(-)
include/linux/mm.h +1 -0
index 74949fb..f7da94d 100644
@@ -1481,6 +1481,7 @@ enum mf_flags {
 extern void memory_failure(unsigned long pfn, int trapno);
 extern int __memory_failure(unsigned long pfn, int trapno, int flags);
 extern int unpoison_memory(unsigned long pfn);
+extern void hwpoison_page_on_free(struct page *p);
 extern int sysctl_memory_failure_early_kill;
 extern int sysctl_memory_failure_recovery;
 extern void shake_page(struct page *p, int access);
include/linux/page-flags.h +14 -1
index 6fa3178..2d4196b 100644
@@ -107,6 +107,9 @@ enum pageflags {
 #endif
 #ifdef CONFIG_MEMORY_FAILURE
 	PG_hwpoison,		/* hardware poisoned page. Don't touch */
+#ifdef CONFIG_HWPOISON_ON_FREE
+	PG_hwpoison_on_free,	/* hwpoison on free */
+#endif
 #endif
 	__NR_PAGEFLAGS,
 
@@ -280,6 +283,15 @@ PAGEFLAG_FALSE(HWPoison)
 #define __PG_HWPOISON 0
 #endif
 
+#if defined(CONFIG_MEMORY_FAILURE) && BITS_PER_LONG == 64
+PAGEFLAG(HWPoisonOnFree, hwpoison_on_free)
+TESTSCFLAG(HWPoisonOnFree, hwpoison_on_free)
+#define __PG_HWPOISON_ON_FREE (1UL << PG_hwpoison_on_free)
+#else
+PAGEFLAG_FALSE(HWPoisonOnFree)
+#define __PG_HWPOISON_ON_FREE 0
+#endif
+
 u64 stable_page_flags(struct page *page);
 
 static inline int PageUptodate(struct page *page)
@@ -406,7 +418,8 @@ static inline void __ClearPageTail(struct page *page)
 	 1 << PG_private | 1 << PG_private_2 | \
 	 1 << PG_buddy	 | 1 << PG_writeback | 1 << PG_reserved | \
 	 1 << PG_slab	 | 1 << PG_swapcache | 1 << PG_active | \
-	 1 << PG_unevictable | __PG_MLOCKED | __PG_HWPOISON)
+	 1 << PG_unevictable | __PG_MLOCKED | __PG_HWPOISON | \
+	 __PG_HWPOISON_ON_FREE)
 
 /*
  * Flags checked when a page is prepped for return by the page allocator.
mm/Kconfig +4 -0
index f0fb912..ede5444 100644
@@ -269,6 +269,10 @@ config MEMORY_FAILURE
 	  even when some of its memory has uncorrected errors. This requires
 	  special hardware support and typically ECC memory.
 
+config HWPOISON_ON_FREE
+	depends on MEMORY_FAILURE && 64BIT
+	def_bool y	
+
 config HWPOISON_INJECT
 	tristate "HWPoison pages injector"
 	depends on MEMORY_FAILURE && DEBUG_KERNEL && PROC_FS
mm/memory-failure.c +45 -1

73 lines not on this page · raw message

mm/page_alloc.c +2 -0
index a8cfa9c..519c24c 100644
@@ -564,6 +564,8 @@ static inline int free_pages_check(struct page *page)
 		(page->mapping != NULL)  |
 		(atomic_read(&page->_count) != 0) |
 		(page->flags & PAGE_FLAGS_CHECK_AT_FREE))) {
+		if (PageHWPoisonOnFree(page))
+			hwpoison_page_on_free(page);
 		bad_page(page);
 		return 1;
 	}
Signature · 7 lines
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#2 lore raw

Re: [PATCH] HWPOISON: Implement hwpoison-on-free for soft offlining

Quoted 13 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Hard offlining was always able to offline most kernel pages by setting the HWPoison flag on it and waiting for the next free. This works on all pages that get eventually freed.

This didn't work for soft offlining unfortunately, because it cannot safely set the HWPoison flag on a still alive page. Handle this by introducing a second page flag HWPoisonOnFree that is handled by page free and enables HWPoison then. This does not add any overhead to page free because it can be handled as a bad flag which are already checked for.

Quoted 2 lines

Since page flags are scarce on 32bit architectures this is only done on 64bit .

I have no problem with the 64bit only page flag.

Quoted 36 lines
---
 include/linux/mm.h         |    1 +
 include/linux/page-flags.h |   15 +++++++++++++-
 mm/Kconfig                 |    4 +++
 mm/memory-failure.c        |   46 +++++++++++++++++++++++++++++++++++++++++++-
 mm/page_alloc.c            |    2 +
 5 files changed, 66 insertions(+), 2 deletions(-)
include/linux/mm.h +1 -0
index 74949fb..f7da94d 100644
@@ -1481,6 +1481,7 @@ enum mf_flags {
 extern void memory_failure(unsigned long pfn, int trapno);
 extern int __memory_failure(unsigned long pfn, int trapno, int flags);
 extern int unpoison_memory(unsigned long pfn);
+extern void hwpoison_page_on_free(struct page *p);
 extern int sysctl_memory_failure_early_kill;
 extern int sysctl_memory_failure_recovery;
 extern void shake_page(struct page *p, int access);
include/linux/page-flags.h +3 -0
index 6fa3178..2d4196b 100644
@@ -107,6 +107,9 @@ enum pageflags {
 #endif
 #ifdef CONFIG_MEMORY_FAILURE
 	PG_hwpoison,		/* hardware poisoned page. Don't touch */
+#ifdef CONFIG_HWPOISON_ON_FREE
+	PG_hwpoison_on_free,	/* hwpoison on free */
+#endif
 #endif
 	__NR_PAGEFLAGS,
 

include/linux/page-flags.h PAGEFLAG_FALSE(HWPoison) · line 286

Quoted 4 lines
include/linux/page-flags.h +1 -0
@@ -280,6 +283,15 @@ PAGEFLAG_FALSE(HWPoison)
 #define __PG_HWPOISON 0
 #endif
 
+#if defined(CONFIG_MEMORY_FAILURE) && BITS_PER_LONG == 64

We have the simpler CONFIG_HWPOISON_ON_FREE :)

include/linux/page-flags.h PAGEFLAG_FALSE(HWPoison) · line 291

Quoted 5 lines
include/linux/page-flags.h +5 -0
+PAGEFLAG(HWPoisonOnFree, hwpoison_on_free)
+TESTSCFLAG(HWPoisonOnFree, hwpoison_on_free)
+#define __PG_HWPOISON_ON_FREE (1UL << PG_hwpoison_on_free)
+#else
+PAGEFLAG_FALSE(HWPoisonOnFree)

Could define SETPAGEFLAG_NOOP(HWPoisonOnFree) too, for eliminating an #ifdef in the .c file.

Quoted 68 lines
include/linux/page-flags.h +5 -1
+#define __PG_HWPOISON_ON_FREE 0
+#endif
+
 u64 stable_page_flags(struct page *page);
 
 static inline int PageUptodate(struct page *page)
@@ -406,7 +418,8 @@ static inline void __ClearPageTail(struct page *page)
 	 1 << PG_private | 1 << PG_private_2 | \
 	 1 << PG_buddy	 | 1 << PG_writeback | 1 << PG_reserved | \
 	 1 << PG_slab	 | 1 << PG_swapcache | 1 << PG_active | \
-	 1 << PG_unevictable | __PG_MLOCKED | __PG_HWPOISON)
+	 1 << PG_unevictable | __PG_MLOCKED | __PG_HWPOISON | \
+	 __PG_HWPOISON_ON_FREE)
 
 /*
  * Flags checked when a page is prepped for return by the page allocator.
mm/Kconfig +4 -0
index f0fb912..ede5444 100644
@@ -269,6 +269,10 @@ config MEMORY_FAILURE
 	  even when some of its memory has uncorrected errors. This requires
 	  special hardware support and typically ECC memory.
 
+config HWPOISON_ON_FREE
+	depends on MEMORY_FAILURE && 64BIT
+	def_bool y	
+
 config HWPOISON_INJECT
 	tristate "HWPoison pages injector"
 	depends on MEMORY_FAILURE && DEBUG_KERNEL && PROC_FS
mm/memory-failure.c +17 -1
index 9c26eec..34901f6 100644
@@ -1268,6 +1268,14 @@ int soft_offline_page(struct page *page, int flags)
 	if (ret == 0)
 		goto done;
 
+#ifdef CONFIG_HWPOISON_ON_FREE
+	/*
+	 * When the page gets freed make sure to poison
+	 * it immediately.
+	 */
+	SetPageHWPoisonOnFree(page);
+#endif
+
 	/*
 	 * Page cache page we can handle?
 	 */
@@ -1279,7 +1287,15 @@ int soft_offline_page(struct page *page, int flags)
 		shake_page(page, 1);
 
 		/*
-		 * Did it turn free?
+		 * Did it get poisoned on free?
+		 */
+		if (PageHWPoison(page)) {
+			pr_info("soft_offline: %#lx: Page freed through shaking\n", pfn);
+			return 0;
+		}
+
+		/* 
+		 * Try to grab it as a free page again.
 		 */
 		ret = get_any_page(page, pfn, 0);
 		if (ret < 0)

mm/memory-failure.c int soft_offline_page(struct page *page, int flags) · line 1310

Quoted 8 lines
mm/memory-failure.c +5 -0
@@ -1287,12 +1303,23 @@ int soft_offline_page(struct page *page, int flags)
 		if (ret == 0)
 			goto done;
 	}
+
+	if (PageHWPoisonOnFree(page)) {
+		pr_info("soft_offline: %#lx: Delaying poision of unknown page %lx to free\n",
+			pfn, page->flags);
+		return -EIO; /* or 0? */

-EIO looks safer because HWPoisonOnFree does not guarantee success.

Quoted 41 lines
mm/memory-failure.c +23 -0
+	}
+
 	if (!PageLRU(page)) {
 		pr_debug("soft_offline: %#lx: unknown non LRU page type %lx\n",
 				pfn, page->flags);
 		return -EIO;
 	}
 
+	/* 
+	 * Normal page cache page here.
+	 */
+
 	lock_page(page);
 	wait_on_page_writeback(page);
 
@@ -1389,3 +1416,20 @@ int is_hwpoison_address(unsigned long addr)
 	return is_hwpoison_entry(entry);
 }
 EXPORT_SYMBOL_GPL(is_hwpoison_address);
+
+#ifdef CONFIG_HWPOISON_ON_FREE
+
+/* 
+ * HWPoison a page after freeing. 
+ * This is the fallback path for most pages we cannot free early.
+ */
+void hwpoison_page_on_free(struct page *p)
+{
+	get_page(p);
+	SetPageHWPoison(p);
+	ClearPageHWPoisonOnFree(p);
+	pr_info("MCE: %#lx: Delayed poisioning of page after free\n",
+		page_to_pfn(p));
+	atomic_long_inc(&mce_bad_pages);
+}
+#endif
mm/page_alloc.c +0 -0
index a8cfa9c..519c24c 100644

mm/page_alloc.c static inline int free_pages_check(struct page *page) · line 568

Quoted 5 lines
mm/page_alloc.c +2 -0
@@ -564,6 +564,8 @@ static inline int free_pages_check(struct page *page)
 		(page->mapping != NULL)  |
 		(atomic_read(&page->_count) != 0) |
 		(page->flags & PAGE_FLAGS_CHECK_AT_FREE))) {
+		if (PageHWPoisonOnFree(page))
+			hwpoison_page_on_free(page);

hwpoison_page_on_free() seems to be undefined when CONFIG_HWPOISON_ON_FREE is not defined.

mm/page_alloc.c static inline int free_pages_check(struct page *page) · line 571

Quoted 3 lines
mm/page_alloc.c +0 -0
 		bad_page(page);
 		return 1;
 	}

Thanks,
Fengguang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#3 lore raw

Re: [PATCH] HWPOISON: Implement hwpoison-on-free for soft offlining

Quoted 5 lines · Wu Fengguang
Quoted 3 lines

+#if defined(CONFIG_MEMORY_FAILURE)&& BITS_PER_LONG == 64

We have the simpler CONFIG_HWPOISON_ON_FREE :)

Leftover from when I didn't have that. Fixed.

Quoted 7 lines

include/linux/page-flags.h PAGEFLAG_FALSE(HWPoison) · line 291

Quoted 5 lines
include/linux/page-flags.h +5 -0
+PAGEFLAG(HWPoisonOnFree, hwpoison_on_free)
+TESTSCFLAG(HWPoisonOnFree, hwpoison_on_free)
+#define __PG_HWPOISON_ON_FREE (1UL<<  PG_hwpoison_on_free)
+#else
+PAGEFLAG_FALSE(HWPoisonOnFree)

Could define SETPAGEFLAG_NOOP(HWPoisonOnFree) too, for eliminating an #ifdef in the .c file.

Ok.

Quoted 9 lines

mm/memory-failure.c int soft_offline_page(struct page *page, int flags) · line 1310

Quoted 7 lines
  	}
mm/memory-failure.c +5 -0
+
+	if (PageHWPoisonOnFree(page)) {
+		pr_info("soft_offline: %#lx: Delaying poision of unknown page %lx to free\n",
+			pfn, page->flags);
+		return -EIO; /* or 0? */

-EIO looks safer because HWPoisonOnFree does not guarantee success.

Ok.

Quoted 13 lines

mm/page_alloc.c static inline int free_pages_check(struct page *page) · line 568

Quoted 11 lines
mm/page_alloc.c +2 -0
index a8cfa9c..519c24c 100644
@@ -564,6 +564,8 @@ static inline int free_pages_check(struct page *page)
  		(page->mapping != NULL)  |
  		(atomic_read(&page->_count) != 0) |
  		(page->flags&  PAGE_FLAGS_CHECK_AT_FREE))) {
+		if (PageHWPoisonOnFree(page))
+			hwpoison_page_on_free(page);

hwpoison_page_on_free() seems to be undefined when CONFIG_HWPOISON_ON_FREE is not defined.

Yes, but I rely on the compiler never generating the call in this case because the test is zero.

It would fail on a unoptimized build, but the kernel doesn't support that anyways.

Thanks for the review.

-Andi

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Reviewed-by
#4 lore raw

Re: [PATCH] HWPOISON: Implement hwpoison-on-free for soft offlining

Quoted 11 lines
Quoted 4 lines

mm/page_alloc.c static inline int free_pages_check(struct page *page) · line 568

Quoted 2 lines
mm/page_alloc.c +2 -0
+		if (PageHWPoisonOnFree(page))
+			hwpoison_page_on_free(page);

hwpoison_page_on_free() seems to be undefined when CONFIG_HWPOISON_ON_FREE is not defined.

Yes, but I rely on the compiler never generating the call in this case because
the test is zero.

It would fail on a unoptimized build, but the kernel doesn't support that anyways.

Fair enough.

Quoted 1 line

Thanks for the review.

:)

Thanks,
Fengguang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#5 lore raw
In reply to Andi Kleen
Quoted 9 lines · Andi Kleen

Here's a somewhat experimental patch to improve soft offlining in hwpoison, but allowing hwpoison on free for not directly freeable page types. It should work for nearly all left over page types that get eventually freed, so this makes soft offlining nearly universal. The only non handleable page types are now pages that never get freed.

Drawback: It needs an additional page flag. Cannot set hwpoison directly because that would not be "soft" and cause errors.

Ping? Any comments on this patch?

Except for the page flag use I think it's nearly a no brainer. A lot of new soft hwpoison capability for very little additional code.

Has anyone a problem using up a 64bit page flag for that?

Thanks,

-Andi

Signature · 7 lines
[email protected] -- Speaking for myself only.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#6 lore raw
Quoted 12 lines
Quoted 9 lines · Andi Kleen

Here's a somewhat experimental patch to improve soft offlining in hwpoison, but allowing hwpoison on free for not directly freeable page types. It should work for nearly all left over page types that get eventually freed, so this makes soft offlining nearly universal. The only non handleable page types are now pages that never get freed.

Drawback: It needs an additional page flag. Cannot set hwpoison directly because that would not be "soft" and cause errors.

Quoted 6 lines

Ping? Any comments on this patch?

Except for the page flag use I think it's nearly a no brainer. A lot of new soft hwpoison capability for very little additional code.

Has anyone a problem using up a 64bit page flag for that?

To me, it's no problem if this keep 64bit only. IOW, I only dislike to add 32bit page flags.

Yeah, memory corruption is very crap and i think your effort has a lot of worth :)

offtopic, I don't think CONFIG_MEMORY_FAILURE and CONFIG_HWPOISON_ON_FREE are symmetric nor easy understandable. can you please consider naming change? (example, CONFIG_HWPOISON/CONFIG_HWPOISON_ON_FREE, CONFIG_MEMORY_FAILURE/CONFIG_MEMORY_FAILURE_SOFT_OFFLINE)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#7 lore raw
Quoted 5 lines

To me, it's no problem if this keep 64bit only. IOW, I only dislike to add 32bit page flags.

Yeah, memory corruption is very crap and i think your effort has a lot of worth :)

Thanks.

Quoted 6 lines

offtopic, I don't think CONFIG_MEMORY_FAILURE and CONFIG_HWPOISON_ON_FREE are symmetric nor easy understandable. can you please consider naming change? (example, CONFIG_HWPOISON/CONFIG_HWPOISON_ON_FREE, CONFIG_MEMORY_FAILURE/CONFIG_MEMORY_FAILURE_SOFT_OFFLINE)

memory-failure was the old name before hwpoison as a term was invented by Andrew.

In theory it would make sense to rename everything to "hwpoison" now. But I decided so far the disadvantages from breaking user configurations and the impact from renaming files far outweight the small benefits in clarity.

So right now I prefer to keep the status quo, but name everything new hwpoison.

-Andi

Signature · 7 lines
[email protected] -- Speaking for myself only.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#8 lore raw
Quoted 18 lines
Quoted 5 lines

To me, it's no problem if this keep 64bit only. IOW, I only dislike to add 32bit page flags.

Yeah, memory corruption is very crap and i think your effort has a lot of worth :)

Thanks.

Quoted 6 lines

offtopic, I don't think CONFIG_MEMORY_FAILURE and CONFIG_HWPOISON_ON_FREE are symmetric nor easy understandable. can you please consider naming change? (example, CONFIG_HWPOISON/CONFIG_HWPOISON_ON_FREE, CONFIG_MEMORY_FAILURE/CONFIG_MEMORY_FAILURE_SOFT_OFFLINE)

memory-failure was the old name before hwpoison as a term was invented by Andrew.

Quoted 7 lines

In theory it would make sense to rename everything to "hwpoison" now. But I decided so far the disadvantages from breaking user configurations and the impact from renaming files far outweight the small benefits in clarity.

So right now I prefer to keep the status quo, but name everything new hwpoison.

ok. I've got it. thanks :)

author
#9 lore raw
In reply to Andi Kleen
Quoted 7 lines · Andi Kleen

Here's a somewhat experimental patch to improve soft offlining in hwpoison, but allowing hwpoison on free for not directly freeable page types. It should work for nearly all left over page types that get eventually freed, so this makes soft offlining nearly universal. The only non handleable page types are now pages that never get freed.

Quoted 6 lines

Drawback: It needs an additional page flag. Cannot set hwpoison directly because that would not be "soft" and cause errors.

Since the flags are scarce on 32bit I only enabled it on 64bit.

Comments?

I got a couple of positive comments and reviews and no negative comments.

So I assume noone has a problem with using up a 64bit page flag for this. I plan to push this into linux-next after some delay and then prepare it for merge later.

If there are any objections please speak up now.

Thanks,
-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.