Skip to contentSkip to conversation

mm: shmem: use proper gfp flags for shmem_writepage()

PATCH In review Yang Shi 3 participants 6 messages Nov 5, 2019 – Nov 7, 2019

Subsystems (2)
lore
Share & tools
lore

Yang Shi

author
#0 lore raw

PATCH 1/1 mm: shmem: use proper gfp flags for shmem_writepage()

The shmem_writepage() uses GFP_ATOMIC to allocate swap cache. GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH | __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should write out to swap only in response to memory pressure, so __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself or in direct reclaim already.

In addition, XArray node allocations from PF_MEMALLOC contexts could completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency reserves from being allocated.

Here just copy the gfp flags used by add_to_swap().

Cc Hugh Dickins <[email protected]>
---
 mm/shmem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/shmem.c +2 -1
index 220be9f..9691dec 100644
@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 	if (list_empty(&info->swaplist))
 		list_add(&info->swaplist, &shmem_swaplist);
 
-	if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
+	if (add_to_swap_cache(page, swap,
+			__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
 		spin_lock_irq(&info->lock);
 		shmem_recalc_inode(inode);
 		info->swapped++;
Signature · 1 line
1.8.3.1
#1 lore raw
Quoted 6 lines · Yang Shi

The shmem_writepage() uses GFP_ATOMIC to allocate swap cache. GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH | __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should write out to swap only in response to memory pressure, so __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself or in direct reclaim already.

What kind of problem are you trying to fix here?

Quoted 3 lines

In addition, XArray node allocations from PF_MEMALLOC contexts could completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency reserves from being allocated.

I am not really familiar with XArray much, could you be more specific please?

Quoted 24 lines

Here just copy the gfp flags used by add_to_swap().

Cc Hugh Dickins <[email protected]>
---
 mm/shmem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/shmem.c +2 -1
index 220be9f..9691dec 100644
@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 	if (list_empty(&info->swaplist))
 		list_add(&info->swaplist, &shmem_swaplist);
 
-	if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
+	if (add_to_swap_cache(page, swap,
+			__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
 		spin_lock_irq(&info->lock);
 		shmem_recalc_inode(inode);
 		info->swapped++;
Signature · 1 line
1.8.3.1
Signature · 2 lines
Michal Hocko
SUSE Labs
author
#2 lore raw
Quoted 8 lines · Michal Hocko
Quoted 6 lines · Yang Shi

The shmem_writepage() uses GFP_ATOMIC to allocate swap cache. GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH | __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should write out to swap only in response to memory pressure, so __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself or in direct reclaim already.

What kind of problem are you trying to fix here?

I didn't run into any visible problem. I just happened to find this inconsistency when I was looking into the other problem.

The add_to_swap() does:

int add_to_swap(struct page *page)
{
...
err = add_to_swap_cache(page, entry, __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
...
}

Actually, shmem_writepage() does almost the same thing and both of them are called in reclaim context, so I didn't see why they should use different gfp flag. And, GFP_ATOMIC is also different from the old definition as I mentioned in the commit log.

Quoted 6 lines
Quoted 4 lines

In addition, XArray node allocations from PF_MEMALLOC contexts could completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency reserves from being allocated.

I am not really familiar with XArray much, could you be more specific please?

It comes from the comments of add_to_swap(), says:

/*
         * XArray node allocations from PF_MEMALLOC contexts could
         * completely exhaust the page allocator. __GFP_NOMEMALLOC
         * stops emergency reserves from being allocated.

And, it looks the original comment came from pre-git time, TBH I'm not quite sure about the specific problem which incurred this. I suspect it may be because PF_MEMALLOC context allows ALLOC_NO_WATERMARK.

Quoted 25 lines
Quoted 25 lines

Here just copy the gfp flags used by add_to_swap().

Cc Hugh Dickins <[email protected]>
---
  mm/shmem.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
mm/shmem.c +2 -1
index 220be9f..9691dec 100644
@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
  	if (list_empty(&info->swaplist))
  		list_add(&info->swaplist, &shmem_swaplist);
 
-	if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
+	if (add_to_swap_cache(page, swap,
+			__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
  		spin_lock_irq(&info->lock);
  		shmem_recalc_inode(inode);
  		info->swapped++;
Signature · 1 line
1.8.3.1
maintainer
#3 lore raw
Quoted 8 lines · Yang Shi
Quoted 7 lines · Michal Hocko
Quoted 6 lines · Yang Shi

The shmem_writepage() uses GFP_ATOMIC to allocate swap cache. GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH | __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should write out to swap only in response to memory pressure, so __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself or in direct reclaim already.

Quoted 4 lines
Quoted 1 line

What kind of problem are you trying to fix here?

I didn't run into any visible problem. I just happened to find this inconsistency when I was looking into the other problem.

Yes, I don't think it fixes any actual problem: just a cleanup to make the two calls look the same when they don't need to be different (whereas the call from __read_swap_cache_async() rightly uses a lower priority gfp).

If it does fix a problem, then you need to worry also about the
* TODO: this could cause a theoretical memory reclaim
* deadlock in the swap out path.
comment still against the call in add_to_swap(): but I think that is equally theoretical, demanding no attention since 2.6.12.

Quoted 59 lines

The add_to_swap() does:

int add_to_swap(struct page *page)
{
...
err = add_to_swap_cache(page, entry, __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
...
}

Actually, shmem_writepage() does almost the same thing and both of them are called in reclaim context, so I didn't see why they should use different gfp flag. And, GFP_ATOMIC is also different from the old definition as I mentioned in the commit log.

Quoted 6 lines
Quoted 4 lines

In addition, XArray node allocations from PF_MEMALLOC contexts could completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency reserves from being allocated.

I am not really familiar with XArray much, could you be more specific please?

It comes from the comments of add_to_swap(), says:

/*
         * XArray node allocations from PF_MEMALLOC contexts could
         * completely exhaust the page allocator. __GFP_NOMEMALLOC
         * stops emergency reserves from being allocated.

And, it looks the original comment came from pre-git time, TBH I'm not quite sure about the specific problem which incurred this. I suspect it may be because PF_MEMALLOC context allows ALLOC_NO_WATERMARK.

Quoted 25 lines
Quoted 25 lines

Here just copy the gfp flags used by add_to_swap().

Cc Hugh Dickins <[email protected]>
---
  mm/shmem.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
mm/shmem.c +2 -0
index 220be9f..9691dec 100644
@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page,
 struct writeback_control *wbc)
  	if (list_empty(&info->swaplist))
  		list_add(&info->swaplist, &shmem_swaplist);
  -	if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
+	if (add_to_swap_cache(page, swap,
+			__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
  		spin_lock_irq(&info->lock);
  		shmem_recalc_inode(inode);
  		info->swapped++;
Signature · 1 line
1.8.3.1
author
#4 lore raw
Quoted 9 lines · Hugh Dickins
Quoted 8 lines · Yang Shi
Quoted 7 lines · Michal Hocko
Quoted 6 lines · Yang Shi

The shmem_writepage() uses GFP_ATOMIC to allocate swap cache. GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH | __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should write out to swap only in response to memory pressure, so __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself or in direct reclaim already.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

What kind of problem are you trying to fix here?

I didn't run into any visible problem. I just happened to find this inconsistency when I was looking into the other problem.

Yes, I don't think it fixes any actual problem: just a cleanup to make the two calls look the same when they don't need to be different (whereas the call from __read_swap_cache_async() rightly uses a lower priority gfp).

I'm supposed it is because __read_swap_cache_async()is typically called from page fault context which is less crucial than reclaim.

Shall I consider this as an ack but with commit log rephrased to reflect the cleanup?

Quoted 62 lines

If it does fix a problem, then you need to worry also about the
* TODO: this could cause a theoretical memory reclaim
* deadlock in the swap out path.
comment still against the call in add_to_swap(): but I think that is equally theoretical, demanding no attention since 2.6.12.

Quoted 55 lines

The add_to_swap() does:

int add_to_swap(struct page *page)
{
...
err = add_to_swap_cache(page, entry,
                        __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
...
}

Actually, shmem_writepage() does almost the same thing and both of them are called in reclaim context, so I didn't see why they should use different gfp flag. And, GFP_ATOMIC is also different from the old definition as I mentioned in the commit log.

Quoted 5 lines
Quoted 3 lines

In addition, XArray node allocations from PF_MEMALLOC contexts could completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency reserves from being allocated.

I am not really familiar with XArray much, could you be more specific please?

It comes from the comments of add_to_swap(), says:

/*
         * XArray node allocations from PF_MEMALLOC contexts could
         * completely exhaust the page allocator. __GFP_NOMEMALLOC
         * stops emergency reserves from being allocated.

And, it looks the original comment came from pre-git time, TBH I'm not quite sure about the specific problem which incurred this. I suspect it may be because PF_MEMALLOC context allows ALLOC_NO_WATERMARK.

Quoted 24 lines
Quoted 24 lines

Here just copy the gfp flags used by add_to_swap().

Cc Hugh Dickins <[email protected]>
---
   mm/shmem.c | 3 ++-
   1 file changed, 2 insertions(+), 1 deletion(-)
mm/shmem.c +2 -0
index 220be9f..9691dec 100644
@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page,
 struct writeback_control *wbc)
   	if (list_empty(&info->swaplist))
   		list_add(&info->swaplist, &shmem_swaplist);
   -	if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
+	if (add_to_swap_cache(page, swap,
+			__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
   		spin_lock_irq(&info->lock);
   		shmem_recalc_inode(inode);
   		info->swapped++;
Signature · 1 line
1.8.3.1
maintainer Acked-by
#5 lore raw
Quoted 8 lines · Yang Shi
Quoted 4 lines · Hugh Dickins

Yes, I don't think it fixes any actual problem: just a cleanup to make the two calls look the same when they don't need to be different (whereas the call from __read_swap_cache_async() rightly uses a lower priority gfp).

I'm supposed it is because __read_swap_cache_async()is typically called from page fault context which is less crucial than reclaim.

Exactly.

Quoted 3 lines

Shall I consider this as an ack but with commit log rephrased to reflect the cleanup?

Okay,