Skip to contentSkip to conversation

mm: hugetlbfs: Add some VM_BUG_ON()s to catch non-hugetlbfs pages

PATCH In review Dave Hansen 4 participants 8 messages Oct 28, 2013 – Nov 6, 2013

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/2 mm: hugetlbfs: Add some VM_BUG_ON()s to catch non-hugetlbfs pages

Dave Jiang reported that he was seeing oopses when running NUMA systems and default_hugepagesz=1G. I traced the issue down to migrate_page_copy() trying to use the same code for hugetlb pages and transparent hugepages. It should not have been trying to pass thp pages in there.

So, add some VM_BUG_ON()s for the next hapless VM developer that tries the same thing.

---

 linux.git-davehans/include/linux/hugetlb.h |    1 +
 linux.git-davehans/mm/hugetlb.c            |    1 +
 2 files changed, 2 insertions(+)
include/linux/hugetlb.h renamed from include/linux/hugetlb.h~bug-not-hugetlbfs-in-copy_huge_page +1 -0
diff -puN include/linux/hugetlb.h~bug-not-hugetlbfs-in-copy_huge_page include/linux/hugetlb.h
@@ -355,6 +355,7 @@ static inline pte_t arch_make_huge_pte(p
 
 static inline struct hstate *page_hstate(struct page *page)
 {
+	VM_BUG_ON(!PageHuge(page));
 	return size_to_hstate(PAGE_SIZE << compound_order(page));
 }
 
mm/hugetlb.c renamed from mm/hugetlb.c~bug-not-hugetlbfs-in-copy_huge_page +1 -0
diff -puN mm/hugetlb.c~bug-not-hugetlbfs-in-copy_huge_page mm/hugetlb.c
@@ -498,6 +498,7 @@ void copy_huge_page(struct page *dst, st
 	int i;
 	struct hstate *h = page_hstate(src);
 
+	VM_BUG_ON(!h);
 	if (unlikely(pages_per_huge_page(h) > MAX_ORDER_NR_PAGES)) {
 		copy_gigantic_page(dst, src);
 		return;

_

#1 lore raw

PATCH 2/2 mm: thp: give transparent hugepage code a separate copy_page

From: Dave Hansen <[email protected]>

Right now, the migration code in migrate_page_copy() uses copy_huge_page() for hugetlbfs and thp pages:

       if (PageHuge(page) || PageTransHuge(page))
                copy_huge_page(newpage, page);

So, yay for code reuse. But:

void copy_huge_page(struct page *dst, struct page *src)

{
        struct hstate *h = page_hstate(src);

and a non-hugetlbfs page has no page_hstate(). This works 99% of the time because page_hstate() determines the hstate from the page order alone. Since the page order of a THP page matches the default hugetlbfs page order, it works.

But, if you change the default huge page size on the boot command-line (say default_hugepagesz=1G), then we might not even *have* a 2MB hstate so page_hstate() returns null and copy_huge_page() oopses pretty fast since copy_huge_page() dereferences the hstate:

void copy_huge_page(struct page *dst, struct page *src)

{
        struct hstate *h = page_hstate(src);
        if (unlikely(pages_per_huge_page(h) > MAX_ORDER_NR_PAGES)) {

...

This patch creates a copy_high_order_page() which can be used on THP pages.

---

 linux.git-davehans/include/linux/huge_mm.h |   16 ++++++++++++++++
 linux.git-davehans/mm/huge_memory.c        |   13 +++++++++++++
 linux.git-davehans/mm/migrate.c            |    4 +++-
 3 files changed, 32 insertions(+), 1 deletion(-)
include/linux/huge_mm.h renamed from include/linux/huge_mm.h~copy-huge-separate-from-copy-transhuge +16 -0
diff -puN include/linux/huge_mm.h~copy-huge-separate-from-copy-transhuge include/linux/huge_mm.h
@@ -177,6 +177,10 @@ static inline struct page *compound_tran
 extern int do_huge_pmd_numa_page(struct mm_struct *mm, struct vm_area_struct *vma,
 				unsigned long addr, pmd_t pmd, pmd_t *pmdp);
 
+extern void copy_high_order_page(struct page *newpage,
+				 struct page *oldpage,
+				 int order);
+
 #else /* CONFIG_TRANSPARENT_HUGEPAGE */
 #define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; })
 #define HPAGE_PMD_MASK ({ BUILD_BUG(); 0; })
@@ -227,6 +231,18 @@ static inline int do_huge_pmd_numa_page(
 	return 0;
 }
 
+/*
+ * The non-stub version of this code is probably usable
+ * generically but its only user is thp at the moment,
+ * so enforce that with a BUG()
+ */
+static inline  void copy_high_order_page(struct page *newpage,
+					 struct page *oldpage,
+					 int order)
+{
+	BUG();
+}
+
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
 
 #endif /* _LINUX_HUGE_MM_H */
mm/huge_memory.c renamed from mm/huge_memory.c~copy-huge-separate-from-copy-transhuge +13 -0
diff -puN mm/huge_memory.c~copy-huge-separate-from-copy-transhuge mm/huge_memory.c
@@ -2789,3 +2789,16 @@ void __vma_adjust_trans_huge(struct vm_a
 			split_huge_page_address(next->vm_mm, nstart);
 	}
 }
+
+void copy_high_order_page(struct page *newpage,
+			  struct page *oldpage,
+			  int order)
+{
+	int i;
+
+	might_sleep();
+	for (i = 0; i < (1<<order); i++) {
+		cond_resched();
+		copy_highpage(newpage + i, oldpage + i);
+	}
+}
mm/migrate.c renamed from mm/migrate.c~copy-huge-separate-from-copy-transhuge +3 -1
diff -puN mm/migrate.c~copy-huge-separate-from-copy-transhuge mm/migrate.c
@@ -443,8 +443,10 @@ int migrate_huge_page_move_mapping(struc
  */
 void migrate_page_copy(struct page *newpage, struct page *page)
 {
-	if (PageHuge(page) || PageTransHuge(page))
+	if (PageHuge(page))
 		copy_huge_page(newpage, page);
+	else if(PageTransHuge(page))
+		copy_high_order_page(newpage, page, HPAGE_PMD_ORDER);
 	else
 		copy_highpage(newpage, page);
 

_

#2 lore raw

Re: [PATCH 2/2] mm: thp: give transparent hugepage code a separate copy_page

Quoted 27 lines · Dave Hansen

From: Dave Hansen <[email protected]>

Right now, the migration code in migrate_page_copy() uses copy_huge_page() for hugetlbfs and thp pages:

       if (PageHuge(page) || PageTransHuge(page))
                copy_huge_page(newpage, page);

So, yay for code reuse. But:

void copy_huge_page(struct page *dst, struct page *src)

{
        struct hstate *h = page_hstate(src);

and a non-hugetlbfs page has no page_hstate(). This works 99% of the time because page_hstate() determines the hstate from the page order alone. Since the page order of a THP page matches the default hugetlbfs page order, it works.

But, if you change the default huge page size on the boot command-line (say default_hugepagesz=1G), then we might not even *have* a 2MB hstate so page_hstate() returns null and copy_huge_page() oopses pretty fast since copy_huge_page() dereferences the hstate:

Quoted 8 lines

void copy_huge_page(struct page *dst, struct page *src)

{
        struct hstate *h = page_hstate(src);
        if (unlikely(pages_per_huge_page(h) > MAX_ORDER_NR_PAGES)) {

...

This patch creates a copy_high_order_page() which can be used on THP pages.

We already have copy_user_huge_page() and copy_user_gigantic_page() in generic code (mm/memory.c). I think copy_gigantic_page() and copy_huge_page() should be moved there too.

BTW, I think pages_per_huge_page in copy_user_huge_page() is redunand:
compound_order(page) should be enough, right?

Signature · 7 lines
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#3 lore raw

Re: [PATCH 2/2] mm: thp: give transparent hugepage code a separatecopy_page

Quoted 34 lines · Kirill A. Shutemov
Quoted 33 lines · Dave Hansen

From: Dave Hansen <[email protected]>

Right now, the migration code in migrate_page_copy() uses copy_huge_page() for hugetlbfs and thp pages:

       if (PageHuge(page) || PageTransHuge(page))
                copy_huge_page(newpage, page);

So, yay for code reuse. But:

void copy_huge_page(struct page *dst, struct page *src)

{
        struct hstate *h = page_hstate(src);

and a non-hugetlbfs page has no page_hstate(). This works 99% of the time because page_hstate() determines the hstate from the page order alone. Since the page order of a THP page matches the default hugetlbfs page order, it works.

But, if you change the default huge page size on the boot command-line (say default_hugepagesz=1G), then we might not even *have* a 2MB hstate so page_hstate() returns null and copy_huge_page() oopses pretty fast since copy_huge_page() dereferences the hstate:

void copy_huge_page(struct page *dst, struct page *src)

{
        struct hstate *h = page_hstate(src);
        if (unlikely(pages_per_huge_page(h) > MAX_ORDER_NR_PAGES)) {

...

Quoted 6 lines
Quoted 2 lines

This patch creates a copy_high_order_page() which can be used on THP pages.

We already have copy_user_huge_page() and copy_user_gigantic_page() in generic code (mm/memory.c). I think copy_gigantic_page() and copy_huge_page() should be moved there too.

I agree this.

Quoted 2 lines

BTW, I think pages_per_huge_page in copy_user_huge_page() is redunand:
compound_order(page) should be enough, right?

I guess that thp code is very strict on performance, so developers chose to pass it as an argument instead of calculating compound_order in each call. I think the performance gain is small (maybe invisible),
but it's not a bad idea to me.

Thanks,
Naoya

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#4 lore raw

Re: [PATCH 2/2] mm: thp: give transparent hugepage code a separate copy_page

In reply to Kirill A. Shutemov
Quoted 7 lines · Kirill A. Shutemov
Quoted 6 lines · Dave Hansen

void copy_huge_page(struct page *dst, struct page *src)

{
        struct hstate *h = page_hstate(src);
        if (unlikely(pages_per_huge_page(h) > MAX_ORDER_NR_PAGES)) {

...

Quoted 6 lines
Quoted 2 lines

This patch creates a copy_high_order_page() which can be used on THP pages.

We already have copy_user_huge_page() and copy_user_gigantic_page() in generic code (mm/memory.c). I think copy_gigantic_page() and copy_huge_page() should be moved there too.

That would be fine I guesss... in another patch. :)

Quoted 2 lines

BTW, I think pages_per_huge_page in copy_user_huge_page() is redunand:
compound_order(page) should be enough, right?

The way it is now, the compiler can optimize for the !HUGETLBFS case. Also, pages_per_huge_page() works for gigantic pages. compound_order() wouldn't work for those.

#5 lore raw

Re: [PATCH 2/2] mm: thp: give transparent hugepage code a separate copy_page

In reply to Dave Hansen

mm/huge_memory.c void __vma_adjust_trans_huge(struct vm_a · line 2804

Quoted 13 lines · Dave Hansen
mm/huge_memory.c renamed from mm/huge_memory.c~copy-huge-separate-from-copy-transhuge +13 -0
+
+void copy_high_order_page(struct page *newpage,
+                         struct page *oldpage,
+                         int order)
+{
+       int i;
+
+       might_sleep();
+       for (i = 0; i < (1<<order); i++) {
+               cond_resched();
+               copy_highpage(newpage + i, oldpage + i);
+       }
+}

Can we make no use of might_sleep here with cond_resched in loop?

#6 lore raw

Re: [PATCH 2/2] mm: thp: give transparent hugepage code a separate copy_page

Quoted 16 lines · Hillf Danton

mm/huge_memory.c void __vma_adjust_trans_huge(struct vm_a · line 2804

Quoted 13 lines · Dave Hansen
mm/huge_memory.c renamed from mm/huge_memory.c~copy-huge-separate-from-copy-transhuge +13 -0
+
+void copy_high_order_page(struct page *newpage,
+                         struct page *oldpage,
+                         int order)
+{
+       int i;
+
+       might_sleep();
+       for (i = 0; i < (1<<order); i++) {
+               cond_resched();
+               copy_highpage(newpage + i, oldpage + i);
+       }
+}

Can we make no use of might_sleep here with cond_resched in loop?

I'm not sure what you're saying.

Are you pointing out that cond_resched() actually calls might_sleep() so the might_sleep() is redundant?

Reviewed-by on v1 1/2
#7 lore raw

Re: [PATCH 1/2] mm: hugetlbfs: Add some VM_BUG_ON()s to catchnon-hugetlbfs pages

In reply to Dave Hansen
Quoted 8 lines · Dave Hansen

Dave Jiang reported that he was seeing oopses when running NUMA systems and default_hugepagesz=1G. I traced the issue down to migrate_page_copy() trying to use the same code for hugetlb pages and transparent hugepages. It should not have been trying to pass thp pages in there.

So, add some VM_BUG_ON()s for the next hapless VM developer that

Looks good to me, thanks.

Quoted 31 lines

---

 linux.git-davehans/include/linux/hugetlb.h |    1 +
 linux.git-davehans/mm/hugetlb.c            |    1 +
 2 files changed, 2 insertions(+)

diff -puN include/linux/hugetlb.h~bug-not-hugetlbfs-in-copy_huge_page include/linux/hugetlb.h

include/linux/hugetlb.h renamed from include/linux/hugetlb.h~bug-not-hugetlbfs-in-copy_huge_page +1 -0
@@ -355,6 +355,7 @@ static inline pte_t arch_make_huge_pte(p
 
 static inline struct hstate *page_hstate(struct page *page)
 {
+	VM_BUG_ON(!PageHuge(page));
 	return size_to_hstate(PAGE_SIZE << compound_order(page));
 }
 

diff -puN mm/hugetlb.c~bug-not-hugetlbfs-in-copy_huge_page mm/hugetlb.c

mm/hugetlb.c renamed from mm/hugetlb.c~bug-not-hugetlbfs-in-copy_huge_page +1 -0
@@ -498,6 +498,7 @@ void copy_huge_page(struct page *dst, st
 	int i;
 	struct hstate *h = page_hstate(src);
 
+	VM_BUG_ON(!h);
 	if (unlikely(pages_per_huge_page(h) > MAX_ORDER_NR_PAGES)) {
 		copy_gigantic_page(dst, src);
 		return;

_

Quoted 6 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>