Skip to contentSkip to conversation

mm: Add PageLayzyFree() helper functions for MADV_FREE

PATCH In review Huang, Ying 3 participants 6 messages Mar 4, 2020 – Mar 9, 2020

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm: Add PageLayzyFree() helper functions for MADV_FREE

From: Huang Ying <[email protected]>

Now PageSwapBacked() is used as the helper function to check whether pages have been freed lazily via MADV_FREE. This isn't very obvious. So Dave suggested to add PageLazyFree() family helper functions to improve the code readability.

Cc David Hildenbrand <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Zi Yan <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Hugh Dickins <[email protected]>

---

Changelog:

v2:

- Avoid code bloat via removing VM_BUG_ON_PAGE(), which doesn't exist in the original code. Now there is no any text/data/bss size change.

- Fix one wrong replacement in try_to_unmap_one().

---
 include/linux/page-flags.h | 25 +++++++++++++++++++++++++
 mm/rmap.c                  |  4 ++--
 mm/swap.c                  | 11 +++--------
 mm/vmscan.c                |  7 +++----
 4 files changed, 33 insertions(+), 14 deletions(-)
include/linux/page-flags.h +25 -0
index 49c2697046b9..192b593750d3 100644
@@ -498,6 +498,31 @@ static __always_inline int PageKsm(struct page *page)
 TESTPAGEFLAG_FALSE(Ksm)
 #endif
 
+/*
+ * For pages freed lazily via MADV_FREE.  lazyfree pages are clean
+ * anonymous pages.  They have SwapBacked flag cleared to distinguish
+ * with normal anonymous pages
+ */
+static __always_inline int __PageLazyFree(struct page *page)
+{
+	return !PageSwapBacked(page);
+}
+
+static __always_inline int PageLazyFree(struct page *page)
+{
+	return PageAnon(page) && __PageLazyFree(page);
+}
+
+static __always_inline void SetPageLazyFree(struct page *page)
+{
+	ClearPageSwapBacked(page);
+}
+
+static __always_inline void ClearPageLazyFree(struct page *page)
+{
+	SetPageSwapBacked(page);
+}
+
 u64 stable_page_flags(struct page *page);
 
 static inline int PageUptodate(struct page *page)
mm/rmap.c +2 -2
index 1c02adaa233e..6ec96c8e7826 100644
@@ -1609,7 +1609,7 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 			}
 
 			/* MADV_FREE page check */
-			if (!PageSwapBacked(page)) {
+			if (__PageLazyFree(page)) {
 				if (!PageDirty(page)) {
 					/* Invalidate as we cleared the pte */
 					mmu_notifier_invalidate_range(mm,
@@ -1623,7 +1623,7 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 				 * discarded. Remap the page to page table.
 				 */
 				set_pte_at(mm, address, pvmw.pte, pteval);
-				SetPageSwapBacked(page);
+				ClearPageLazyFree(page);
 				ret = false;
 				page_vma_mapped_walk_done(&pvmw);
 				break;
mm/swap.c +3 -8
index c1d3ca80ea10..d83f2cd4cdb8 100644
@@ -563,7 +563,7 @@ static void lru_deactivate_fn(struct page *page, struct lruvec *lruvec,
 static void lru_lazyfree_fn(struct page *page, struct lruvec *lruvec,
 			    void *arg)
 {
-	if (PageLRU(page) && PageAnon(page) && PageSwapBacked(page) &&
+	if (PageLRU(page) && PageAnon(page) && !__PageLazyFree(page) &&
 	    !PageSwapCache(page) && !PageUnevictable(page)) {
 		bool active = PageActive(page);
 
@@ -571,12 +571,7 @@ static void lru_lazyfree_fn(struct page *page, struct lruvec *lruvec,
 				       LRU_INACTIVE_ANON + active);
 		ClearPageActive(page);
 		ClearPageReferenced(page);
-		/*
-		 * lazyfree pages are clean anonymous pages. They have
-		 * SwapBacked flag cleared to distinguish normal anonymous
-		 * pages
-		 */
-		ClearPageSwapBacked(page);
+		SetPageLazyFree(page);
 		add_page_to_lru_list(page, lruvec, LRU_INACTIVE_FILE);
 
 		__count_vm_events(PGLAZYFREE, hpage_nr_pages(page));
@@ -678,7 +673,7 @@ void deactivate_page(struct page *page)
  */
 void mark_page_lazyfree(struct page *page)
 {
-	if (PageLRU(page) && PageAnon(page) && PageSwapBacked(page) &&
+	if (PageLRU(page) && PageAnon(page) && !__PageLazyFree(page) &&
 	    !PageSwapCache(page) && !PageUnevictable(page)) {
 		struct pagevec *pvec = &get_cpu_var(lru_lazyfree_pvecs);
 
mm/vmscan.c +3 -4
index eca49a1c2f68..40bb41ada2d2 100644
@@ -1043,8 +1043,7 @@ static void page_check_dirty_writeback(struct page *page,
 	 * Anonymous pages are not handled by flushers and must be written
 	 * from reclaim context. Do not stall reclaim based on them
 	 */
-	if (!page_is_file_cache(page) ||
-	    (PageAnon(page) && !PageSwapBacked(page))) {
+	if (!page_is_file_cache(page) || PageLazyFree(page)) {
 		*dirty = false;
 		*writeback = false;
 		return;
@@ -1235,7 +1234,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 		 * Try to allocate it some swap space here.
 		 * Lazyfree page could be freed directly
 		 */
-		if (PageAnon(page) && PageSwapBacked(page)) {
+		if (PageAnon(page) && !__PageLazyFree(page)) {
 			if (!PageSwapCache(page)) {
 				if (!(sc->gfp_mask & __GFP_IO))
 					goto keep_locked;
@@ -1411,7 +1410,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 			}
 		}
 
-		if (PageAnon(page) && !PageSwapBacked(page)) {
+		if (PageLazyFree(page)) {
 			/* follow __remove_mapping for reference */
 			if (!page_ref_freeze(page, 1))
 				goto keep_locked;
Signature · 1 line
2.25.0
#1 lore raw
Quoted 41 lines · Ying

From: Huang Ying <[email protected]>

Now PageSwapBacked() is used as the helper function to check whether pages have been freed lazily via MADV_FREE. This isn't very obvious. So Dave suggested to add PageLazyFree() family helper functions to improve the code readability.

Cc David Hildenbrand <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Zi Yan <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Hugh Dickins <[email protected]>

---

Changelog:

v2:

- Avoid code bloat via removing VM_BUG_ON_PAGE(), which doesn't exist in the original code. Now there is no any text/data/bss size change.

- Fix one wrong replacement in try_to_unmap_one().

---
 include/linux/page-flags.h | 25 +++++++++++++++++++++++++
 mm/rmap.c                  |  4 ++--
 mm/swap.c                  | 11 +++--------
 mm/vmscan.c                |  7 +++----
 4 files changed, 33 insertions(+), 14 deletions(-)
include/linux/page-flags.h +0 -0
index 49c2697046b9..192b593750d3 100644

include/linux/page-flags.h static __always_inline int PageKsm(struct page *page) · line 504

Quoted 7 lines
include/linux/page-flags.h +4 -0
@@ -498,6 +498,31 @@ static __always_inline int PageKsm(struct page *page)
 TESTPAGEFLAG_FALSE(Ksm)
 #endif
 
+/*
+ * For pages freed lazily via MADV_FREE.  lazyfree pages are clean
+ * anonymous pages.  They have SwapBacked flag cleared to distinguish
+ * with normal anonymous pages

"They don't have PG_swapbacked set, to distinguish them from normal anonymous pages." ?

Quoted 51 lines
include/linux/page-flags.h +21 -0
+ */
+static __always_inline int __PageLazyFree(struct page *page)
+{
+	return !PageSwapBacked(page);
+}
+
+static __always_inline int PageLazyFree(struct page *page)
+{
+	return PageAnon(page) && __PageLazyFree(page);
+}
+
+static __always_inline void SetPageLazyFree(struct page *page)
+{
+	ClearPageSwapBacked(page);
+}
+
+static __always_inline void ClearPageLazyFree(struct page *page)
+{
+	SetPageSwapBacked(page);
+}
+
 u64 stable_page_flags(struct page *page);
 
 static inline int PageUptodate(struct page *page)
mm/rmap.c +2 -2
index 1c02adaa233e..6ec96c8e7826 100644
@@ -1609,7 +1609,7 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 			}
 
 			/* MADV_FREE page check */
-			if (!PageSwapBacked(page)) {
+			if (__PageLazyFree(page)) {
 				if (!PageDirty(page)) {
 					/* Invalidate as we cleared the pte */
 					mmu_notifier_invalidate_range(mm,
@@ -1623,7 +1623,7 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 				 * discarded. Remap the page to page table.
 				 */
 				set_pte_at(mm, address, pvmw.pte, pteval);
-				SetPageSwapBacked(page);
+				ClearPageLazyFree(page);
 				ret = false;
 				page_vma_mapped_walk_done(&pvmw);
 				break;
mm/swap.c +0 -0
index c1d3ca80ea10..d83f2cd4cdb8 100644

mm/swap.c static void lru_deactivate_fn(struct page *page, struct lruvec *lruvec, · line 566

Quoted 5 lines
mm/swap.c +1 -1
@@ -563,7 +563,7 @@ static void lru_deactivate_fn(struct page *page, struct lruvec *lruvec,
 static void lru_lazyfree_fn(struct page *page, struct lruvec *lruvec,
 			    void *arg)
 {
-	if (PageLRU(page) && PageAnon(page) && PageSwapBacked(page) &&
+	if (PageLRU(page) && PageAnon(page) && !__PageLazyFree(page) &&

See my comment below

Quoted 18 lines
mm/swap.c +1 -6
 	    !PageSwapCache(page) && !PageUnevictable(page)) {
 		bool active = PageActive(page);
 
@@ -571,12 +571,7 @@ static void lru_lazyfree_fn(struct page *page, struct lruvec *lruvec,
 				       LRU_INACTIVE_ANON + active);
 		ClearPageActive(page);
 		ClearPageReferenced(page);
-		/*
-		 * lazyfree pages are clean anonymous pages. They have
-		 * SwapBacked flag cleared to distinguish normal anonymous
-		 * pages
-		 */
-		ClearPageSwapBacked(page);
+		SetPageLazyFree(page);
 		add_page_to_lru_list(page, lruvec, LRU_INACTIVE_FILE);
 
 		__count_vm_events(PGLAZYFREE, hpage_nr_pages(page));

mm/swap.c void deactivate_page(struct page *page) · line 676

Quoted 5 lines
mm/swap.c +1 -1
@@ -678,7 +673,7 @@ void deactivate_page(struct page *page)
  */
 void mark_page_lazyfree(struct page *page)
 {
-	if (PageLRU(page) && PageAnon(page) && PageSwapBacked(page) &&
+	if (PageLRU(page) && PageAnon(page) && !__PageLazyFree(page) &&

See my comment below.

Quoted 18 lines
mm/swap.c +0 -0
 	    !PageSwapCache(page) && !PageUnevictable(page)) {
 		struct pagevec *pvec = &get_cpu_var(lru_lazyfree_pvecs);
 
mm/vmscan.c +1 -2
index eca49a1c2f68..40bb41ada2d2 100644
@@ -1043,8 +1043,7 @@ static void page_check_dirty_writeback(struct page *page,
 	 * Anonymous pages are not handled by flushers and must be written
 	 * from reclaim context. Do not stall reclaim based on them
 	 */
-	if (!page_is_file_cache(page) ||
-	    (PageAnon(page) && !PageSwapBacked(page))) {
+	if (!page_is_file_cache(page) || PageLazyFree(page)) {
 		*dirty = false;
 		*writeback = false;
 		return;

mm/vmscan.c static unsigned long shrink_page_list(struct list_head *page_list, · line 1237

Quoted 5 lines
mm/vmscan.c +1 -1
@@ -1235,7 +1234,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 		 * Try to allocate it some swap space here.
 		 * Lazyfree page could be freed directly
 		 */
-		if (PageAnon(page) && PageSwapBacked(page)) {
+		if (PageAnon(page) && !__PageLazyFree(page)) {

I don't think this chunk makes this code easier to understand. I'd just keep this as is. IOW, I prefer PageSwapBacked() over !__PageLazyFree().

In general, I don't think this patch really improves the situation ... it's only a handful of places where this change slightly makes the code easier to understand. And there, only slightly ... I'd prefer better comments instead (e.g., in PageAnon()), documenting what it means for a anon page to either have PageSwapBacked() set or not.

Signature · 3 lines
Thanks,

David / dhildenb
#2 lore raw
Quoted 6 lines · David Hildenbrand

In general, I don't think this patch really improves the situation ... it's only a handful of places where this change slightly makes the code easier to understand. And there, only slightly ... I'd prefer better comments instead (e.g., in PageAnon()), documenting what it means for a anon page to either have PageSwapBacked() set or not.

Agreed, I think any changes to clarify what PageSwapBacked means when it's set and when it's clear for PageAnon should be in the form of a comment, likely in page-flags.h. That's currently lacking for lazy free pages.

#3 lore raw
In reply to David Hildenbrand
Quoted 45 lines · David Hildenbrand
Quoted 44 lines · Ying

From: Huang Ying <[email protected]>

Now PageSwapBacked() is used as the helper function to check whether pages have been freed lazily via MADV_FREE. This isn't very obvious. So Dave suggested to add PageLazyFree() family helper functions to improve the code readability.

Cc David Hildenbrand <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Zi Yan <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Hugh Dickins <[email protected]>

---

Changelog:

v2:

- Avoid code bloat via removing VM_BUG_ON_PAGE(), which doesn't exist in the original code. Now there is no any text/data/bss size change.

- Fix one wrong replacement in try_to_unmap_one().

---
 include/linux/page-flags.h | 25 +++++++++++++++++++++++++
 mm/rmap.c                  |  4 ++--
 mm/swap.c                  | 11 +++--------
 mm/vmscan.c                |  7 +++----
 4 files changed, 33 insertions(+), 14 deletions(-)
include/linux/page-flags.h +0 -0
index 49c2697046b9..192b593750d3 100644
@@ -498,6 +498,31 @@ static __always_inline int PageKsm(struct page *page)
 TESTPAGEFLAG_FALSE(Ksm)
 #endif
 
Quoted 7 lines

include/linux/page-flags.h static __always_inline int PageKsm(struct page *page) · line 504

Quoted 4 lines
include/linux/page-flags.h +4 -0
+/*
+ * For pages freed lazily via MADV_FREE.  lazyfree pages are clean
+ * anonymous pages.  They have SwapBacked flag cleared to distinguish
+ * with normal anonymous pages

"They don't have PG_swapbacked set, to distinguish them from normal anonymous pages." ?

Thanks!

Quoted 113 lines
Quoted 51 lines
include/linux/page-flags.h +21 -0
+ */
+static __always_inline int __PageLazyFree(struct page *page)
+{
+	return !PageSwapBacked(page);
+}
+
+static __always_inline int PageLazyFree(struct page *page)
+{
+	return PageAnon(page) && __PageLazyFree(page);
+}
+
+static __always_inline void SetPageLazyFree(struct page *page)
+{
+	ClearPageSwapBacked(page);
+}
+
+static __always_inline void ClearPageLazyFree(struct page *page)
+{
+	SetPageSwapBacked(page);
+}
+
 u64 stable_page_flags(struct page *page);
 
 static inline int PageUptodate(struct page *page)
mm/rmap.c +2 -2
index 1c02adaa233e..6ec96c8e7826 100644
@@ -1609,7 +1609,7 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 			}
 
 			/* MADV_FREE page check */
-			if (!PageSwapBacked(page)) {
+			if (__PageLazyFree(page)) {
 				if (!PageDirty(page)) {
 					/* Invalidate as we cleared the pte */
 					mmu_notifier_invalidate_range(mm,
@@ -1623,7 +1623,7 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 				 * discarded. Remap the page to page table.
 				 */
 				set_pte_at(mm, address, pvmw.pte, pteval);
-				SetPageSwapBacked(page);
+				ClearPageLazyFree(page);
 				ret = false;
 				page_vma_mapped_walk_done(&pvmw);
 				break;
mm/swap.c +0 -0
index c1d3ca80ea10..d83f2cd4cdb8 100644

mm/swap.c static void lru_deactivate_fn(struct page *page, struct lruvec *lruvec, · line 566

Quoted 5 lines
mm/swap.c +1 -1
@@ -563,7 +563,7 @@ static void lru_deactivate_fn(struct page *page, struct lruvec *lruvec,
 static void lru_lazyfree_fn(struct page *page, struct lruvec *lruvec,
 			    void *arg)
 {
-	if (PageLRU(page) && PageAnon(page) && PageSwapBacked(page) &&
+	if (PageLRU(page) && PageAnon(page) && !__PageLazyFree(page) &&

See my comment below

Quoted 18 lines
mm/swap.c +1 -6
 	    !PageSwapCache(page) && !PageUnevictable(page)) {
 		bool active = PageActive(page);
 
@@ -571,12 +571,7 @@ static void lru_lazyfree_fn(struct page *page, struct lruvec *lruvec,
 				       LRU_INACTIVE_ANON + active);
 		ClearPageActive(page);
 		ClearPageReferenced(page);
-		/*
-		 * lazyfree pages are clean anonymous pages. They have
-		 * SwapBacked flag cleared to distinguish normal anonymous
-		 * pages
-		 */
-		ClearPageSwapBacked(page);
+		SetPageLazyFree(page);
 		add_page_to_lru_list(page, lruvec, LRU_INACTIVE_FILE);
 
 		__count_vm_events(PGLAZYFREE, hpage_nr_pages(page));

mm/swap.c void deactivate_page(struct page *page) · line 676

Quoted 5 lines
mm/swap.c +1 -1
@@ -678,7 +673,7 @@ void deactivate_page(struct page *page)
  */
 void mark_page_lazyfree(struct page *page)
 {
-	if (PageLRU(page) && PageAnon(page) && PageSwapBacked(page) &&
+	if (PageLRU(page) && PageAnon(page) && !__PageLazyFree(page) &&

See my comment below.

Quoted 18 lines
mm/swap.c +0 -0
 	    !PageSwapCache(page) && !PageUnevictable(page)) {
 		struct pagevec *pvec = &get_cpu_var(lru_lazyfree_pvecs);
 
mm/vmscan.c +1 -2
index eca49a1c2f68..40bb41ada2d2 100644
@@ -1043,8 +1043,7 @@ static void page_check_dirty_writeback(struct page *page,
 	 * Anonymous pages are not handled by flushers and must be written
 	 * from reclaim context. Do not stall reclaim based on them
 	 */
-	if (!page_is_file_cache(page) ||
-	    (PageAnon(page) && !PageSwapBacked(page))) {
+	if (!page_is_file_cache(page) || PageLazyFree(page)) {
 		*dirty = false;
 		*writeback = false;
 		return;

mm/vmscan.c static unsigned long shrink_page_list(struct list_head *page_list, · line 1237

Quoted 5 lines
mm/vmscan.c +1 -1
@@ -1235,7 +1234,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 		 * Try to allocate it some swap space here.
 		 * Lazyfree page could be freed directly
 		 */
-		if (PageAnon(page) && PageSwapBacked(page)) {
+		if (PageAnon(page) && !__PageLazyFree(page)) {

I don't think this chunk makes this code easier to understand. I'd just keep this as is. IOW, I prefer PageSwapBacked() over !__PageLazyFree().

Quoted 5 lines

In general, I don't think this patch really improves the situation ... it's only a handful of places where this change slightly makes the code easier to understand. And there, only slightly ... I'd prefer better comments instead (e.g., in PageAnon()), documenting what it means for a anon page to either have PageSwapBacked() set or not.

Personally, I still prefer the better named functions than the comments here and there. But I can understand that people may have different flavor.

Best Regards,
Huang, Ying

#4 lore raw
Quoted 10 lines · Ying
Quoted 5 lines

In general, I don't think this patch really improves the situation ... it's only a handful of places where this change slightly makes the code easier to understand. And there, only slightly ... I'd prefer better comments instead (e.g., in PageAnon()), documenting what it means for a anon page to either have PageSwapBacked() set or not.

Personally, I still prefer the better named functions than the comments here and there. But I can understand that people may have different flavor.

Maybe add a comment to page-flags.h referring to what PageSwapBacked indicates when PageAnon is true?

#5 lore raw
Quoted 8 lines · David Rientjes
Quoted 6 lines · Ying
Quoted 5 lines

In general, I don't think this patch really improves the situation ... it's only a handful of places where this change slightly makes the code easier to understand. And there, only slightly ... I'd prefer better comments instead (e.g., in PageAnon()), documenting what it means for a anon page to either have PageSwapBacked() set or not.

Quoted 7 lines
Quoted 4 lines

Personally, I still prefer the better named functions than the comments here and there. But I can understand that people may have different flavor.

Maybe add a comment to page-flags.h referring to what PageSwapBacked indicates when PageAnon is true?

If someone find a confusing PageSwapBacked() invocation, and if we only want to use comments to resolve the confusing, the best place to add the comments is above the line where PageSwapBacked() is invoked. Because it's harder for people to dig out the right comments in page-flags.h. The appropriate named helper functions can replace that comments and be more elegant.

Best Regards,
Huang, Ying