mm: shmem: use proper gfp flags for shmem_writepage()
Subsystems (2)
PATCH 1/1 mm: shmem: use proper gfp flags for shmem_writepage()
The shmem_writepage() uses GFP_ATOMIC to allocate swap cache. GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH | __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should write out to swap only in response to memory pressure, so __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself or in direct reclaim already.
In addition, XArray node allocations from PF_MEMALLOC contexts could completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency reserves from being allocated.
Here just copy the gfp flags used by add_to_swap().
--- mm/shmem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
mm/shmem.c +2 -1
@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)if (list_empty (&info->swaplist))list_add (&info->swaplist, &shmem_swaplist); -if (add_to_swap_cache (page, swap,GFP_ATOMIC ) ==0 ) { +if (add_to_swap_cache (page, swap, + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) ==0 ) {spin_lock_irq (&info->lock);shmem_recalc_inode (inode); info->swapped++;
Signature · 1 line
1.8.3.1
Quoted 6 lines · Yang Shi
The
shmem_writepage()usesGFP_ATOMICto allocate swap cache.GFP_ATOMICused to mean__GFP_HIGH, but now it means__GFP_HIGH|__GFP_ATOMIC|__GFP_KSWAPD_RECLAIM. However,shmem_writepage()should write out to swap only in response to memory pressure, so__GFP_KSWAPD_RECLAIMlooks useless since the caller may be kswapd itself or in direct reclaim already.
What kind of problem are you trying to fix here?
Quoted 3 lines
In addition, XArray node allocations from
PF_MEMALLOCcontexts could completely exhaust the page allocator,__GFP_NOMEMALLOCstops emergency reserves from being allocated.
I am not really familiar with XArray much, could you be more specific please?
Quoted 24 lines
Here just copy the gfp flags used by
add_to_swap().Cc Hugh Dickins <[email protected]>Signed-off-by Yang Shi <[email protected]>--- mm/shmem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/shmem.c +2
-1@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)if (list_empty (&info->swaplist))list_add (&info->swaplist, &shmem_swaplist); -if (add_to_swap_cache (page, swap,GFP_ATOMIC ) ==0 ) { +if (add_to_swap_cache (page, swap, + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) ==0 ) {spin_lock_irq (&info->lock);shmem_recalc_inode (inode); info->swapped++;Signature · 1 line
1.8.3.1
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 8 lines · Michal Hocko
Quoted 6 lines · Yang Shi
The
shmem_writepage()usesGFP_ATOMICto allocate swap cache.GFP_ATOMICused to mean__GFP_HIGH, but now it means__GFP_HIGH|__GFP_ATOMIC|__GFP_KSWAPD_RECLAIM. However,shmem_writepage()should write out to swap only in response to memory pressure, so__GFP_KSWAPD_RECLAIMlooks useless since the caller may be kswapd itself or in direct reclaim already.What kind of problem are you trying to fix here?
I didn't run into any visible problem. I just happened to find this inconsistency when I was looking into the other problem.
The add_to_swap() does:
int add_to_swap(struct page *page)
{
...
err = add_to_swap_cache(page, entry, __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
...
}
Actually, shmem_writepage() does almost the same thing and both of them are called in reclaim context, so I didn't see why they should use different gfp flag. And, GFP_ATOMIC is also different from the old definition as I mentioned in the commit log.
Quoted 6 lines
Quoted 4 lines
In addition, XArray node allocations from
PF_MEMALLOCcontexts could completely exhaust the page allocator,__GFP_NOMEMALLOCstops emergency reserves from being allocated.I am not really familiar with XArray much, could you be more specific please?
It comes from the comments of add_to_swap(), says:
/*
* XArray node allocations from PF_MEMALLOC contexts could
* completely exhaust the page allocator. __GFP_NOMEMALLOC
* stops emergency reserves from being allocated.
And, it looks the original comment came from pre-git time, TBH I'm not quite sure about the specific problem which incurred this. I suspect it may be because PF_MEMALLOC context allows ALLOC_NO_WATERMARK.
Quoted 25 lines
Quoted 25 lines
Here just copy the gfp flags used by
add_to_swap().Cc Hugh Dickins <[email protected]>Signed-off-by Yang Shi <[email protected]>--- mm/shmem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/shmem.c +2
-1@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)if (list_empty (&info->swaplist))list_add (&info->swaplist, &shmem_swaplist); -if (add_to_swap_cache (page, swap,GFP_ATOMIC ) ==0 ) { +if (add_to_swap_cache (page, swap, + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) ==0 ) {spin_lock_irq (&info->lock);shmem_recalc_inode (inode); info->swapped++;Signature · 1 line
1.8.3.1
Hugh Dickins
maintainerQuoted 8 lines · Yang Shi
Quoted 7 lines · Michal Hocko
Quoted 6 lines · Yang Shi
The
shmem_writepage()usesGFP_ATOMICto allocate swap cache.GFP_ATOMICused to mean__GFP_HIGH, but now it means__GFP_HIGH|__GFP_ATOMIC|__GFP_KSWAPD_RECLAIM. However,shmem_writepage()should write out to swap only in response to memory pressure, so__GFP_KSWAPD_RECLAIMlooks useless since the caller may be kswapd itself or in direct reclaim already.
Quoted 4 lines
Quoted 1 line
What kind of problem are you trying to fix here?
I didn't run into any visible problem. I just happened to find this inconsistency when I was looking into the other problem.
Yes, I don't think it fixes any actual problem: just a cleanup to make the two calls look the same when they don't need to be different (whereas the call from __read_swap_cache_async() rightly uses a lower priority gfp).
If it does fix a problem, then you need to worry also about the
* TODO: this could cause a theoretical memory reclaim
* deadlock in the swap out path.
comment still against the call in add_to_swap(): but I think that is equally theoretical, demanding no attention since 2.6.12.
Quoted 59 lines
The
add_to_swap()does:int
add_to_swap(struct page *page)
{
...
err =add_to_swap_cache(page, entry, __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
...
}Actually,
shmem_writepage()does almost the same thing and both of them are called in reclaim context, so I didn't see why they should use different gfp flag. And,GFP_ATOMICis also different from the old definition as I mentioned in the commit log.Quoted 6 lines
Quoted 4 lines
In addition, XArray node allocations from
PF_MEMALLOCcontexts could completely exhaust the page allocator,__GFP_NOMEMALLOCstops emergency reserves from being allocated.I am not really familiar with XArray much, could you be more specific please?
It comes from the comments of
add_to_swap(), says:/*
* XArray node allocations fromPF_MEMALLOCcontexts could
* completely exhaust the page allocator.__GFP_NOMEMALLOC
* stops emergency reserves from being allocated.And, it looks the original comment came from pre-git time, TBH I'm not quite sure about the specific problem which incurred this. I suspect it may be because
PF_MEMALLOCcontext allowsALLOC_NO_WATERMARK.Quoted 25 lines
Quoted 25 lines
Here just copy the gfp flags used by
add_to_swap().Cc Hugh Dickins <[email protected]>Signed-off-by Yang Shi <[email protected]>--- mm/shmem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/shmem.c +2
-0@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)if (list_empty (&info->swaplist))list_add (&info->swaplist, &shmem_swaplist); - if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) { +if (add_to_swap_cache (page, swap, + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) ==0 ) {spin_lock_irq (&info->lock);shmem_recalc_inode (inode); info->swapped++;Signature · 1 line
1.8.3.1
Quoted 9 lines · Hugh Dickins
Quoted 8 lines · Yang Shi
Quoted 7 lines · Michal Hocko
Quoted 6 lines · Yang Shi
The
shmem_writepage()usesGFP_ATOMICto allocate swap cache.GFP_ATOMICused to mean__GFP_HIGH, but now it means__GFP_HIGH|__GFP_ATOMIC|__GFP_KSWAPD_RECLAIM. However,shmem_writepage()should write out to swap only in response to memory pressure, so__GFP_KSWAPD_RECLAIMlooks useless since the caller may be kswapd itself or in direct reclaim already.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
What kind of problem are you trying to fix here?
I didn't run into any visible problem. I just happened to find this inconsistency when I was looking into the other problem.
Yes, I don't think it fixes any actual problem: just a cleanup to make the two calls look the same when they don't need to be different (whereas the call from
__read_swap_cache_async()rightly uses a lower priority gfp).
I'm supposed it is because __read_swap_cache_async()is typically called from page fault context which is less crucial than reclaim.
Shall I consider this as an ack but with commit log rephrased to reflect the cleanup?
Quoted 62 lines
If it does fix a problem, then you need to worry also about the
* TODO: this could cause a theoretical memory reclaim
* deadlock in the swap out path.
comment still against the call inadd_to_swap(): but I think that is equally theoretical, demanding no attention since 2.6.12.Quoted 55 lines
The
add_to_swap()does:int
add_to_swap(struct page *page)
{
...
err = add_to_swap_cache(page, entry,
__GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
...
}Actually,
shmem_writepage()does almost the same thing and both of them are called in reclaim context, so I didn't see why they should use different gfp flag. And,GFP_ATOMICis also different from the old definition as I mentioned in the commit log.Quoted 5 lines
Quoted 3 lines
In addition, XArray node allocations from
PF_MEMALLOCcontexts could completely exhaust the page allocator,__GFP_NOMEMALLOCstops emergency reserves from being allocated.I am not really familiar with XArray much, could you be more specific please?
It comes from the comments of
add_to_swap(), says:/*
* XArray node allocations fromPF_MEMALLOCcontexts could
* completely exhaust the page allocator.__GFP_NOMEMALLOC
* stops emergency reserves from being allocated.And, it looks the original comment came from pre-git time, TBH I'm not quite sure about the specific problem which incurred this. I suspect it may be because
PF_MEMALLOCcontext allowsALLOC_NO_WATERMARK.Quoted 24 lines
Quoted 24 lines
Here just copy the gfp flags used by
add_to_swap().Cc Hugh Dickins <[email protected]>Signed-off-by Yang Shi <[email protected]>--- mm/shmem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/shmem.c +2
-0@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)if (list_empty (&info->swaplist))list_add (&info->swaplist, &shmem_swaplist); - if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) { +if (add_to_swap_cache (page, swap, + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) ==0 ) {spin_lock_irq (&info->lock);shmem_recalc_inode (inode); info->swapped++;Signature · 1 line
1.8.3.1
Quoted 8 lines · Yang Shi
Quoted 4 lines · Hugh Dickins
Yes, I don't think it fixes any actual problem: just a cleanup to make the two calls look the same when they don't need to be different (whereas the call from
__read_swap_cache_async()rightly uses a lower priority gfp).I'm supposed it is because __read_swap_cache_async()is typically called from page fault context which is less crucial than reclaim.
Exactly.
Quoted 3 lines
Shall I consider this as an ack but with commit log rephrased to reflect the cleanup?
Okay,