Skip to contentSkip to conversation

mm: fix is_pinnable_page against on cma page

PATCH Superseded Minchan Kim 3 participants 3 messages May 5, 2022 – May 8, 2022

Subsystems (2)

v2 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm: fix is_pinnable_page against on cma page

Pages on CMA area could have MIGRATE_ISOLATE as well as MIGRATE_CMA so current is_pinnable_page could miss CMA pages which has MIGRATE_ ISOLATE. It ends up putting CMA pages longterm pinning possible on pin_user_pages APIs so CMA allocation fails.

The CMA allocation path protects the migration type change race using zone->lock but what GUP path need to know is just whether the page is on CMA area or not rather than exact type. Thus, we don't need zone->lock but just checks the migratype in either of (MIGRATE_ISOLATE and MIGRATE_CMA).

Adding the MIGRATE_ISOLATE check in is_pinnable_page could cause rejecting of pinning the page on MIGRATE_ISOLATE pageblock even thouth it's neither CMA nor movable zone if the page is temporarily unmovable. However, the migration failure is general issue, not only come from MIGRATE_ISOLATE and the MIGRATE_ISOLATE is also transient state like other temporal refcount holding of pages.

Cc David Hildenbrand <[email protected]>

---
* from v1 - https://lore.kernel.org/all/[email protected]/
* fix build warning - lkp
* fix refetching issue of migration type
* add side effect on !ZONE_MOVABLE and !MIGRATE_CMA in description - david

 include/linux/mm.h | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
include/linux/mm.h +12 -2
index 6acca5cecbc5..e77758e2035e 100644
@@ -1625,8 +1625,18 @@ static inline bool page_needs_cow_for_dma(struct vm_area_struct *vma,
 #ifdef CONFIG_MIGRATION
 static inline bool is_pinnable_page(struct page *page)
 {
-	return !(is_zone_movable_page(page) || is_migrate_cma_page(page)) ||
-		is_zero_pfn(page_to_pfn(page));
+#ifdef CONFIG_CMA
+	/*
+	 * use volatile to use local variable mt instead of
+	 * refetching mt value.
+	 */
+	volatile int mt = get_pageblock_migratetype(page);
+
+	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
+		return false;
+#endif
+
+	return !(is_zone_movable_page(page) || is_zero_pfn(page_to_pfn(page)));
 }
 #else
 static inline bool is_pinnable_page(struct page *page)
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
maintainer
#1 lore raw
Quoted 11 lines · On Wed, 4 May 2022 23:44:29 -0700 Minchan Kim <[email protected]> wrote:

Pages on CMA area could have MIGRATE_ISOLATE as well as MIGRATE_CMA so current is_pinnable_page could miss CMA pages which has MIGRATE_ ISOLATE. It ends up putting CMA pages longterm pinning possible on pin_user_pages APIs so CMA allocation fails.

The CMA allocation path protects the migration type change race using zone->lock but what GUP path need to know is just whether the page is on CMA area or not rather than exact type. Thus, we don't need zone->lock but just checks the migratype in either of (MIGRATE_ISOLATE and MIGRATE_CMA).

Quoted 3 lines

Adding the MIGRATE_ISOLATE check in is_pinnable_page could cause rejecting of pinning the page on MIGRATE_ISOLATE pageblock even thouth it's neither CMA nor movable zone if the page is temporarily

"though"

Quoted 14 lines

unmovable. However, the migration failure is general issue, not only come from MIGRATE_ISOLATE and the MIGRATE_ISOLATE is also transient state like other temporal refcount holding of pages.

...

include/linux/mm.h +0 -2
@@ -1625,8 +1625,18 @@ static inline bool page_needs_cow_for_dma(struct vm_area_struct *vma,
 #ifdef CONFIG_MIGRATION
 static inline bool is_pinnable_page(struct page *page)
 {
-	return !(is_zone_movable_page(page) || is_migrate_cma_page(page)) ||
-		is_zero_pfn(page_to_pfn(page));

include/linux/mm.h static inline bool page_needs_cow_for_dma(struct vm_area_struct *vma, · line 1637

Quoted 10 lines
include/linux/mm.h +10 -0
+#ifdef CONFIG_CMA
+	/*
+	 * use volatile to use local variable mt instead of
+	 * refetching mt value.
+	 */
+	volatile int mt = get_pageblock_migratetype(page);
+
+	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
+		return false;
+#endif

Open-coded use of `volatile' draws unwelcome attention.

What are we trying to do here? Prevent the compiler from rerunning all of get_pageblock_migratetype() (really __get_pfnblock_flags_mask()) twice? That would be pretty dumb of it?

Would a suitably-commented something like

	int __mt = get_pageblock_migratetype(page);
	int mt = __READ_ONCE(__mt);

express this better?

Quoted 5 lines
include/linux/mm.h +2 -0
+
+	return !(is_zone_movable_page(page) || is_zero_pfn(page_to_pfn(page)));
 }
 #else
 static inline bool is_pinnable_page(struct page *page)
#2 lore raw
Quoted 50 lines · Andrew Morton
Quoted 11 lines · On Wed, 4 May 2022 23:44:29 -0700 Minchan Kim <[email protected]> wrote:

Pages on CMA area could have MIGRATE_ISOLATE as well as MIGRATE_CMA so current is_pinnable_page could miss CMA pages which has MIGRATE_ ISOLATE. It ends up putting CMA pages longterm pinning possible on pin_user_pages APIs so CMA allocation fails.

The CMA allocation path protects the migration type change race using zone->lock but what GUP path need to know is just whether the page is on CMA area or not rather than exact type. Thus, we don't need zone->lock but just checks the migratype in either of (MIGRATE_ISOLATE and MIGRATE_CMA).

Quoted 3 lines

Adding the MIGRATE_ISOLATE check in is_pinnable_page could cause rejecting of pinning the page on MIGRATE_ISOLATE pageblock even thouth it's neither CMA nor movable zone if the page is temporarily

"though"

Quoted 14 lines

unmovable. However, the migration failure is general issue, not only come from MIGRATE_ISOLATE and the MIGRATE_ISOLATE is also transient state like other temporal refcount holding of pages.

...

include/linux/mm.h +0 -2
@@ -1625,8 +1625,18 @@ static inline bool page_needs_cow_for_dma(struct vm_area_struct *vma,
 #ifdef CONFIG_MIGRATION
 static inline bool is_pinnable_page(struct page *page)
 {
-	return !(is_zone_movable_page(page) || is_migrate_cma_page(page)) ||
-		is_zero_pfn(page_to_pfn(page));

include/linux/mm.h static inline bool page_needs_cow_for_dma(struct vm_area_struct *vma, · line 1637

Quoted 10 lines
include/linux/mm.h +10 -0
+#ifdef CONFIG_CMA
+	/*
+	 * use volatile to use local variable mt instead of
+	 * refetching mt value.
+	 */
+	volatile int mt = get_pageblock_migratetype(page);
+
+	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
+		return false;
+#endif

Open-coded use of `volatile' draws unwelcome attention.

What are we trying to do here? Prevent the compiler from rerunning all of get_pageblock_migratetype() (really __get_pfnblock_flags_mask()) twice? That would be pretty dumb of it?

Quoted 6 lines

Would a suitably-commented something like

	int __mt = get_pageblock_migratetype(page);
	int mt = __READ_ONCE(__mt);

express this better?

Yes, we want READ_ONCE I think. Apart from that LGTM.

Signature · 3 lines
Thanks,

David / dhildenb