Skip to contentSkip to conversation

mm/hugetlb: allow gigantic page allocation to migrate away smaller huge page

PATCH In review Pingfan Liu 3 participants 6 messages Jun 24, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/hugetlb: allow gigantic page allocation to migrate away smaller huge page

The current pfn_range_valid_gigantic() rejects the pud huge page allocation if there is a pmd huge page inside the candidate range.

But pud huge resource is more rare, which should align on 1GB on x86. It is worth to allow migrating away pmd huge page to make room for a pud huge page.

The same logic is applied to pgd and pud huge pages.

Cc Mike Kravetz <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Andrew Morton <[email protected]>
---
 mm/hugetlb.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
mm/hugetlb.c +5 -3
index ac843d3..02d1978 100644
@@ -1081,7 +1081,11 @@ static bool pfn_range_valid_gigantic(struct zone *z,
 			unsigned long start_pfn, unsigned long nr_pages)
 {
 	unsigned long i, end_pfn = start_pfn + nr_pages;
-	struct page *page;
+	struct page *page = pfn_to_page(start_pfn);
+
+	if (PageHuge(page))
+		if (compound_order(compound_head(page)) >= nr_pages)
+			return false;
 
 	for (i = start_pfn; i < end_pfn; i++) {
 		if (!pfn_valid(i))
@@ -1098,8 +1102,6 @@ static bool pfn_range_valid_gigantic(struct zone *z,
 		if (page_count(page) > 0)
 			return false;
 
-		if (PageHuge(page))
-			return false;
 	}
 
 	return true;
Signature · 1 line
2.7.5
#1 lore raw
Quoted 8 lines · Pingfan Liu

The current pfn_range_valid_gigantic() rejects the pud huge page allocation if there is a pmd huge page inside the candidate range.

But pud huge resource is more rare, which should align on 1GB on x86. It is worth to allow migrating away pmd huge page to make room for a pud huge page.

The same logic is applied to pgd and pud huge pages.

I'm sorry but I don't quite understand why we should do this. Is this a bug or an optimization? It sounds like an optimization.

Quoted 16 lines
Cc Mike Kravetz <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Andrew Morton <[email protected]>
---
 mm/hugetlb.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
mm/hugetlb.c +0 -0
index ac843d3..02d1978 100644

mm/hugetlb.c static bool pfn_range_valid_gigantic(struct zone *z, · line 1087

Quoted 8 lines
mm/hugetlb.c +4 -1
@@ -1081,7 +1081,11 @@ static bool pfn_range_valid_gigantic(struct zone *z,
 			unsigned long start_pfn, unsigned long nr_pages)
 {
 	unsigned long i, end_pfn = start_pfn + nr_pages;
-	struct page *page;
+	struct page *page = pfn_to_page(start_pfn);
+
+	if (PageHuge(page))
+		if (compound_order(compound_head(page)) >= nr_pages)

I don't think you want compound_order() here.

Ira

Quoted 16 lines
mm/hugetlb.c +1 -2
+			return false;
 
 	for (i = start_pfn; i < end_pfn; i++) {
 		if (!pfn_valid(i))
@@ -1098,8 +1102,6 @@ static bool pfn_range_valid_gigantic(struct zone *z,
 		if (page_count(page) > 0)
 			return false;
 
-		if (PageHuge(page))
-			return false;
 	}
 
 	return true;
Signature · 1 line
2.7.5
#2 lore raw
Quoted 5 lines · Ira Weiny
Quoted 3 lines · Pingfan Liu

The current pfn_range_valid_gigantic() rejects the pud huge page allocation if there is a pmd huge page inside the candidate range.

Quoted 8 lines
Quoted 5 lines

But pud huge resource is more rare, which should align on 1GB on x86. It is worth to allow migrating away pmd huge page to make room for a pud huge page.

The same logic is applied to pgd and pud huge pages.

I'm sorry but I don't quite understand why we should do this. Is this a bug or an optimization? It sounds like an optimization.

Yes, an optimization. It can help us to success to allocate a 1GB hugetlb if there is some 2MB hugetlb sit in the candidate range. Allocation 1GB hugetlb requires more tough condition, not only a continuous 1GB range, but also aligned on GB. While allocating a 2MB range is easier.

Quoted 12 lines
Quoted 12 lines
Cc Mike Kravetz <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Andrew Morton <[email protected]>
---
 mm/hugetlb.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
Quoted 15 lines

mm/hugetlb.c static bool pfn_range_valid_gigantic(struct zone *z, · line 1087

Quoted 13 lines
mm/hugetlb.c +4 -1
index ac843d3..02d1978 100644
@@ -1081,7 +1081,11 @@ static bool pfn_range_valid_gigantic(struct zone *z,
                      unsigned long start_pfn, unsigned long nr_pages)
 {
      unsigned long i, end_pfn = start_pfn + nr_pages;
-     struct page *page;
+     struct page *page = pfn_to_page(start_pfn);
+
+     if (PageHuge(page))
+             if (compound_order(compound_head(page)) >= nr_pages)

I don't think you want compound_order() here.

Yes, your are right.

Thanks,
Pingfan

Quoted 19 lines

Ira

Quoted 16 lines
mm/hugetlb.c +1 -3
+                     return false;
 
      for (i = start_pfn; i < end_pfn; i++) {
              if (!pfn_valid(i))
@@ -1098,8 +1102,6 @@ static bool pfn_range_valid_gigantic(struct zone *z,
              if (page_count(page) > 0)
                      return false;
 
-             if (PageHuge(page))
-                     return false;
      }
 
      return true;
--

2.7.5

#3 lore raw
In reply to Pingfan Liu
Quoted 8 lines · Pingfan Liu

The current pfn_range_valid_gigantic() rejects the pud huge page allocation if there is a pmd huge page inside the candidate range.

But pud huge resource is more rare, which should align on 1GB on x86. It is worth to allow migrating away pmd huge page to make room for a pud huge page.

The same logic is applied to pgd and pud huge pages.

The huge page in the range can either be a THP or HugeTLB and migrating them has different costs and chances of success. THP migration will involve splitting if THP migration is not enabled and all related TLB related costs. Are you sure that a PUD HugeTLB allocation really should go through these ? Is there any guarantee that after migration of multiple PMD sized THP/HugeTLB pages on the given range, the allocation request for PUD will succeed ?

Quoted 30 lines
Cc Mike Kravetz <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Andrew Morton <[email protected]>
---
 mm/hugetlb.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
mm/hugetlb.c +5 -1
index ac843d3..02d1978 100644
@@ -1081,7 +1081,11 @@ static bool pfn_range_valid_gigantic(struct zone *z,
 			unsigned long start_pfn, unsigned long nr_pages)
 {
 	unsigned long i, end_pfn = start_pfn + nr_pages;
-	struct page *page;
+	struct page *page = pfn_to_page(start_pfn);
+
+	if (PageHuge(page))
+		if (compound_order(compound_head(page)) >= nr_pages)
+			return false;
 
 	for (i = start_pfn; i < end_pfn; i++) {
 		if (!pfn_valid(i))

mm/hugetlb.c static bool pfn_range_valid_gigantic(struct zone *z, · line 1107

Quoted 8 lines
mm/hugetlb.c +0 -2
@@ -1098,8 +1102,6 @@ static bool pfn_range_valid_gigantic(struct zone *z,
 		if (page_count(page) > 0)
 			return false;
 
-		if (PageHuge(page))
-			return false;
 	}
 
 	return true;

So except in the case where there is a bigger huge page in the range this will attempt migrating everything on the way. As mentioned before if it all this is a good idea, it needs to differentiate between HugeTLB and THP and also take into account costs of migrations and chance of subsequence allocation attempt into account.

#4 lore raw
Quoted 11 lines · Anshuman Khandual
Quoted 7 lines · Pingfan Liu

The current pfn_range_valid_gigantic() rejects the pud huge page allocation if there is a pmd huge page inside the candidate range.

But pud huge resource is more rare, which should align on 1GB on x86. It is worth to allow migrating away pmd huge page to make room for a pud huge page.

Quoted 6 lines
Quoted 1 line

The same logic is applied to pgd and pud huge pages.

The huge page in the range can either be a THP or HugeTLB and migrating them has different costs and chances of success. THP migration will involve splitting if THP migration is not enabled and all related TLB related costs. Are you sure that a PUD HugeTLB allocation really should go through these ? Is there any

PUD hugetlb has already driven out PMD thp in current. This patch just want to make PUD hugetlb survives PMD hugetlb.

Quoted 2 lines

guarantee that after migration of multiple PMD sized THP/HugeTLB pages on the given range, the allocation request for PUD will succeed ?

The migration is complicated, but as my understanding, if there is no gup pin in the range and there is enough memory including swap, then it will success.

Quoted 37 lines
Quoted 37 lines
Cc Mike Kravetz <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Andrew Morton <[email protected]>
---
 mm/hugetlb.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
mm/hugetlb.c +5 -3
index ac843d3..02d1978 100644
@@ -1081,7 +1081,11 @@ static bool pfn_range_valid_gigantic(struct zone *z,
                      unsigned long start_pfn, unsigned long nr_pages)
 {
      unsigned long i, end_pfn = start_pfn + nr_pages;
-     struct page *page;
+     struct page *page = pfn_to_page(start_pfn);
+
+     if (PageHuge(page))
+             if (compound_order(compound_head(page)) >= nr_pages)
+                     return false;
 
      for (i = start_pfn; i < end_pfn; i++) {
              if (!pfn_valid(i))
@@ -1098,8 +1102,6 @@ static bool pfn_range_valid_gigantic(struct zone *z,
              if (page_count(page) > 0)
                      return false;
 
-             if (PageHuge(page))
-                     return false;
      }
 
Quoted 8 lines

mm/hugetlb.c static bool pfn_range_valid_gigantic(struct zone *z, · line 1107

Quoted 2 lines
mm/hugetlb.c +0 -0
      return true;

So except in the case where there is a bigger huge page in the range this will attempt migrating everything on the way. As mentioned before if it all this is a good idea, it needs to differentiate between HugeTLB and THP and also take into account costs of migrations and chance of subsequence allocation attempt into account.

Sorry, but I think this logic is only for hugetlb. The caller alloc_gigantic_page() is only used inside mm/hugetlb.c, not by huge_memory.c.

Thanks,
Pingfan

#5 lore raw
Quoted 13 lines · Pingfan Liu
Quoted 11 lines · Anshuman Khandual
Quoted 7 lines · Pingfan Liu

The current pfn_range_valid_gigantic() rejects the pud huge page allocation if there is a pmd huge page inside the candidate range.

But pud huge resource is more rare, which should align on 1GB on x86. It is worth to allow migrating away pmd huge page to make room for a pud huge page.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

The same logic is applied to pgd and pud huge pages.

The huge page in the range can either be a THP or HugeTLB and migrating them has different costs and chances of success. THP migration will involve splitting if THP migration is not enabled and all related TLB related costs. Are you sure that a PUD HugeTLB allocation really should go through these ? Is there any

PUD hugetlb has already driven out PMD thp in current. This patch just want to make PUD hugetlb survives PMD hugetlb.

You are right. PageHuge() is true only for HugeTLB pages unlike PageTransHuge() which is true for both HugeTLB and THP pages. So the current code does migrate the THP out in order to allocate a gigantic HugeTLB. While here just wondering should not we exclude THP as well unless it supports ARCH_HAS_THP_MIGRATION.