mm: fix draining remote pageset
Subsystems (2)
PATCH 1/1 mm: fix draining remote pageset
If there is no memory allocation/freeing in the remote pageset after some time (3 seconds for now), the remote pageset will be drained to avoid memory wastage.
But in the current implementation, vmstat updater worker may not be re-queued when we are waiting for the timeout (pcp->expire != 0) if there are no vmstat changes, for example, when CPU goes idle.
This is fixed via guaranteeing that the vmstat updater worker will always be re-queued when we are waiting for the timeout.
We can reproduce the bug via allocating/freeing pages from remote node, then go idle. And the patch can fix it.
--- mm/vmstat.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
mm/vmstat.c +3 -1
@@ -856,8 +856,10 @@ static int refresh_cpu_vm_stats(bool do_pagesets)continue ; } -if (__this_cpu_dec_return (pcp->expire)) +if (__this_cpu_dec_return (pcp->expire)) { + changes++;continue ; + }if (__this_cpu_read (pcp->count)) {drain_zone_pages (zone,this_cpu_ptr (pcp));
Signature · 1 line
2.39.2
Michal Hocko
reviewerQuoted 7 lines · Huang Ying
If there is no memory allocation/freeing in the remote pageset after some time (3 seconds for now), the remote pageset will be drained to avoid memory wastage.
But in the current implementation, vmstat updater worker may not be re-queued when we are waiting for the timeout (
pcp->expire!= 0) if there are no vmstat changes, for example, when CPU goes idle.
Why is that a problem?
Quoted 34 lines
This is fixed via guaranteeing that the vmstat updater worker will always be re-queued when we are waiting for the timeout.
We can reproduce the bug via allocating/freeing pages from remote node, then go idle. And the patch can fix it.
Fixes 7cc36bbddde5 ("vmstat: on-demand vmstat workers V8")Signed-off-by "Huang, Ying" <[email protected]>Cc Christoph Lameter <[email protected]>Cc Mel Gorman <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Michal Hocko <[email protected]>--- mm/vmstat.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)mm/vmstat.c +3
-1@@ -856,8 +856,10 @@ static int refresh_cpu_vm_stats(bool do_pagesets)continue ; } -if (__this_cpu_dec_return (pcp->expire)) +if (__this_cpu_dec_return (pcp->expire)) { + changes++;continue ; + }if (__this_cpu_read (pcp->count)) {drain_zone_pages (zone,this_cpu_ptr (pcp));Signature · 1 line
2.39.2
Signature · 2 lines
Michal Hocko SUSE Labs
Hi, Michal,
Quoted 10 lines · Michal Hocko
Quoted 7 lines · Huang Ying
If there is no memory allocation/freeing in the remote pageset after some time (3 seconds for now), the remote pageset will be drained to avoid memory wastage.
But in the current implementation, vmstat updater worker may not be re-queued when we are waiting for the timeout (
pcp->expire!= 0) if there are no vmstat changes, for example, when CPU goes idle.Why is that a problem?
The pages of the remote zone may be kept in the local per-CPU pageset for long time as long as there's no page allocation/freeing on the logical CPU. In addition to the logical CPU goes idle, this is also possible if the logical CPU is busy in the user space.
I will update the change log to include this.
--
Best Regards,
Huang, Ying
Quoted 34 lines
Quoted 34 lines
This is fixed via guaranteeing that the vmstat updater worker will always be re-queued when we are waiting for the timeout.
We can reproduce the bug via allocating/freeing pages from remote node, then go idle. And the patch can fix it.
Fixes 7cc36bbddde5 ("vmstat: on-demand vmstat workers V8")Signed-off-by "Huang, Ying" <[email protected]>Cc Christoph Lameter <[email protected]>Cc Mel Gorman <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Michal Hocko <[email protected]>--- mm/vmstat.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)mm/vmstat.c +3
-1@@ -856,8 +856,10 @@ static int refresh_cpu_vm_stats(bool do_pagesets)continue ; } -if (__this_cpu_dec_return (pcp->expire)) +if (__this_cpu_dec_return (pcp->expire)) { + changes++;continue ; + }if (__this_cpu_read (pcp->count)) {drain_zone_pages (zone,this_cpu_ptr (pcp));Signature · 1 line
2.39.2
Michal Hocko
reviewerQuoted 13 lines · Ying
Hi, Michal,
Quoted 9 lines · Michal Hocko
Quoted 7 lines · Huang Ying
If there is no memory allocation/freeing in the remote pageset after some time (3 seconds for now), the remote pageset will be drained to avoid memory wastage.
But in the current implementation, vmstat updater worker may not be re-queued when we are waiting for the timeout (
pcp->expire!= 0) if there are no vmstat changes, for example, when CPU goes idle.
Quoted 6 lines
Quoted 1 line
Why is that a problem?
The pages of the remote zone may be kept in the local per-CPU pageset for long time as long as there's no page allocation/freeing on the logical CPU. In addition to the logical CPU goes idle, this is also possible if the logical CPU is busy in the user space.
But why is this a problem? Is the scale of the problem sufficient to trigger out of memory situations or be otherwise harmful?
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 16 lines · Michal Hocko
Quoted 15 lines · Ying
Hi, Michal,
Quoted 10 lines · Michal Hocko
Quoted 7 lines · Huang Ying
If there is no memory allocation/freeing in the remote pageset after some time (3 seconds for now), the remote pageset will be drained to avoid memory wastage.
But in the current implementation, vmstat updater worker may not be re-queued when we are waiting for the timeout (
pcp->expire!= 0) if there are no vmstat changes, for example, when CPU goes idle.Why is that a problem?
Quoted 7 lines
Quoted 4 lines
The pages of the remote zone may be kept in the local per-CPU pageset for long time as long as there's no page allocation/freeing on the logical CPU. In addition to the logical CPU goes idle, this is also possible if the logical CPU is busy in the user space.
But why is this a problem? Is the scale of the problem sufficient to trigger out of memory situations or be otherwise harmful?
This may trigger premature page reclaiming. The pages in the PCP of the remote zone would have been freed to satisfy the page allocation for the remote zone to avoid page reclaiming. It's highly possible that the local CPU just allocate/free from/to the remote zone temporarily. So, we should free PCP pages of the remote zone if there is no page allocation/freeing from/to the remote zone for 3 seconds.
This will not trigger OOM, because all PCP will be drained if allocation failed after direct reclaiming.
--
Best Regards,
Huang, Ying
Quoted 18 lines · Ying
Quoted 9 lines · Michal Hocko
Quoted 6 lines
Quoted 1 line
Why is that a problem?
The pages of the remote zone may be kept in the local per-CPU pageset for long time as long as there's no page allocation/freeing on the logical CPU. In addition to the logical CPU goes idle, this is also possible if the logical CPU is busy in the user space.
But why is this a problem? Is the scale of the problem sufficient to trigger out of memory situations or be otherwise harmful?
This may trigger premature page reclaiming. The pages in the PCP of the remote zone would have been freed to satisfy the page allocation for the remote zone to avoid page reclaiming. It's highly possible that the local CPU just allocate/free from/to the remote zone temporarily. So, we should free PCP pages of the remote zone if there is no page allocation/freeing from/to the remote zone for 3 seconds.
Quoted 2 lines
This will not trigger OOM, because all PCP will be drained if allocation failed after direct reclaiming.
I think this is a minor issue but its cleaner behavior. vmstat refresh's should continue as long as there are pages queued in remote pcps.
Michal Hocko
reviewerQuoted 23 lines · Ying
Quoted 21 lines · Michal Hocko
Quoted 19 lines · Ying
Hi, Michal,
Quoted 10 lines · Michal Hocko
Quoted 7 lines · Huang Ying
If there is no memory allocation/freeing in the remote pageset after some time (3 seconds for now), the remote pageset will be drained to avoid memory wastage.
But in the current implementation, vmstat updater worker may not be re-queued when we are waiting for the timeout (
pcp->expire!= 0) if there are no vmstat changes, for example, when CPU goes idle.Why is that a problem?
The pages of the remote zone may be kept in the local per-CPU pageset for long time as long as there's no page allocation/freeing on the logical CPU. In addition to the logical CPU goes idle, this is also possible if the logical CPU is busy in the user space.
Quoted 7 lines
Quoted 2 lines
But why is this a problem? Is the scale of the problem sufficient to trigger out of memory situations or be otherwise harmful?
This may trigger premature page reclaiming. The pages in the PCP of the remote zone would have been freed to satisfy the page allocation for the remote zone to avoid page reclaiming. It's highly possible that the local CPU just allocate/free from/to the remote zone temporarily.
I am slightly confused here but I suspect by zone you mean remote pcp. But more importantly is this a concern seen in real workload? Can you quantify it in some manner? E.g. with this patch we have X more kswapd scanning or even hit direct reclaim much less often.
Quoted 3 lines
So,
we should free PCP pages of the remote zone if there is no page allocation/freeing from/to the remote zone for 3 seconds.
Well, I would argue this depends a lot. There are workloads which really like to have CPUs idle and yet they would like to benefit from the allocator fast path after that CPU goes out of idle because idling is their power saving opportunity while workloads want to act quickly after there is something to run.
That being said, we really need some numbers (ideally from real world) that proves this is not just a theoretical concern.
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 40 lines · Michal Hocko
Quoted 23 lines · Ying
Quoted 21 lines · Michal Hocko
Quoted 19 lines · Ying
Hi, Michal,
Quoted 10 lines · Michal Hocko
Quoted 7 lines · Huang Ying
If there is no memory allocation/freeing in the remote pageset after some time (3 seconds for now), the remote pageset will be drained to avoid memory wastage.
But in the current implementation, vmstat updater worker may not be re-queued when we are waiting for the timeout (
pcp->expire!= 0) if there are no vmstat changes, for example, when CPU goes idle.Why is that a problem?
The pages of the remote zone may be kept in the local per-CPU pageset for long time as long as there's no page allocation/freeing on the logical CPU. In addition to the logical CPU goes idle, this is also possible if the logical CPU is busy in the user space.
Quoted 7 lines
Quoted 2 lines
But why is this a problem? Is the scale of the problem sufficient to trigger out of memory situations or be otherwise harmful?
This may trigger premature page reclaiming. The pages in the PCP of the remote zone would have been freed to satisfy the page allocation for the remote zone to avoid page reclaiming. It's highly possible that the local CPU just allocate/free from/to the remote zone temporarily.
I am slightly confused here but I suspect by zone you mean remote pcp. But more importantly is this a concern seen in real workload? Can you quantify it in some manner? E.g. with this patch we have X more kswapd scanning or even hit direct reclaim much less often.
Quoted 3 lines
So,
we should free PCP pages of the remote zone if there is no page allocation/freeing from/to the remote zone for 3 seconds.
Quoted 8 lines
Well, I would argue this depends a lot. There are workloads which really like to have CPUs idle and yet they would like to benefit from the allocator fast path after that CPU goes out of idle because idling is their power saving opportunity while workloads want to act quickly after there is something to run.
That being said, we really need some numbers (ideally from real world) that proves this is not just a theoretical concern.
The behavior to drain the PCP of the remote zone (that is, remote PCP) was introduced in commit 4ae7c03943fc ("[PATCH] Periodically drain non local pagesets"). The goal of draining was well documented in the change log. IIUC, some of your questions can be answered there?
This patch just restores the original behavior changed by commit 7cc36bbddde5 ("vmstat: on-demand vmstat workers V8").
--
Best Regards,
Huang, Ying
Michal Hocko
reviewerQuoted 51 lines · Ying
Quoted 40 lines · Michal Hocko
Quoted 23 lines · Ying
Quoted 21 lines · Michal Hocko
Quoted 19 lines · Ying
Hi, Michal,
Quoted 10 lines · Michal Hocko
Quoted 7 lines · Huang Ying
If there is no memory allocation/freeing in the remote pageset after some time (3 seconds for now), the remote pageset will be drained to avoid memory wastage.
But in the current implementation, vmstat updater worker may not be re-queued when we are waiting for the timeout (
pcp->expire!= 0) if there are no vmstat changes, for example, when CPU goes idle.Why is that a problem?
The pages of the remote zone may be kept in the local per-CPU pageset for long time as long as there's no page allocation/freeing on the logical CPU. In addition to the logical CPU goes idle, this is also possible if the logical CPU is busy in the user space.
Quoted 7 lines
Quoted 2 lines
But why is this a problem? Is the scale of the problem sufficient to trigger out of memory situations or be otherwise harmful?
This may trigger premature page reclaiming. The pages in the PCP of the remote zone would have been freed to satisfy the page allocation for the remote zone to avoid page reclaiming. It's highly possible that the local CPU just allocate/free from/to the remote zone temporarily.
I am slightly confused here but I suspect by zone you mean remote pcp. But more importantly is this a concern seen in real workload? Can you quantify it in some manner? E.g. with this patch we have X more kswapd scanning or even hit direct reclaim much less often.
Quoted 3 lines
So,
we should free PCP pages of the remote zone if there is no page allocation/freeing from/to the remote zone for 3 seconds.Quoted 8 lines
Well, I would argue this depends a lot. There are workloads which really like to have CPUs idle and yet they would like to benefit from the allocator fast path after that CPU goes out of idle because idling is their power saving opportunity while workloads want to act quickly after there is something to run.
That being said, we really need some numbers (ideally from real world) that proves this is not just a theoretical concern.
Quoted 7 lines
The behavior to drain the PCP of the remote zone (that is, remote PCP) was introduced in commit
4ae7c03943fc("[PATCH] Periodically drain non local pagesets"). The goal of draining was well documented in the change log. IIUC, some of your questions can be answered there?This patch just restores the original behavior changed by commit
7cc36bbddde5("vmstat: on-demand vmstat workers V8").
Let me repeat. You need some numbers to show this is needed.
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 57 lines · Michal Hocko
Quoted 56 lines · Ying
Quoted 40 lines · Michal Hocko
Quoted 23 lines · Ying
Quoted 21 lines · Michal Hocko
Quoted 19 lines · Ying
Hi, Michal,
Quoted 10 lines · Michal Hocko
Quoted 7 lines · Huang Ying
If there is no memory allocation/freeing in the remote pageset after some time (3 seconds for now), the remote pageset will be drained to avoid memory wastage.
But in the current implementation, vmstat updater worker may not be re-queued when we are waiting for the timeout (
pcp->expire!= 0) if there are no vmstat changes, for example, when CPU goes idle.Why is that a problem?
The pages of the remote zone may be kept in the local per-CPU pageset for long time as long as there's no page allocation/freeing on the logical CPU. In addition to the logical CPU goes idle, this is also possible if the logical CPU is busy in the user space.
Quoted 7 lines
Quoted 2 lines
But why is this a problem? Is the scale of the problem sufficient to trigger out of memory situations or be otherwise harmful?
This may trigger premature page reclaiming. The pages in the PCP of the remote zone would have been freed to satisfy the page allocation for the remote zone to avoid page reclaiming. It's highly possible that the local CPU just allocate/free from/to the remote zone temporarily.
I am slightly confused here but I suspect by zone you mean remote pcp. But more importantly is this a concern seen in real workload? Can you quantify it in some manner? E.g. with this patch we have X more kswapd scanning or even hit direct reclaim much less often.
Quoted 3 lines
So,
we should free PCP pages of the remote zone if there is no page allocation/freeing from/to the remote zone for 3 seconds.Quoted 8 lines
Well, I would argue this depends a lot. There are workloads which really like to have CPUs idle and yet they would like to benefit from the allocator fast path after that CPU goes out of idle because idling is their power saving opportunity while workloads want to act quickly after there is something to run.
That being said, we really need some numbers (ideally from real world) that proves this is not just a theoretical concern.
The behavior to drain the PCP of the remote zone (that is, remote PCP) was introduced in commit
4ae7c03943fc("[PATCH] Periodically drain non local pagesets"). The goal of draining was well documented in the change log. IIUC, some of your questions can be answered there?
Quoted 4 lines
Quoted 2 lines
This patch just restores the original behavior changed by commit
7cc36bbddde5("vmstat: on-demand vmstat workers V8").Let me repeat. You need some numbers to show this is needed.
I have done some test for this patch as follows,
- Run some workloads, use numactl to bind CPU to node 0 and memory to node 1. So the PCP of the CPU on node 0 for zone on node 1 will be filled.
- After workloads finish, idle for 60s
- Check /proc/zoneinfo
With the original kernel, the number of pages in the PCP of the CPU on node 0 for zone on node 1 is non-zero after idle. With the patched kernel, that becomes 0 after idle. We avoid to keep pages in the remote PCP during idle.
This is the number I have. If you think it isn't enough to justify the patch, then I'm OK too (although I think it's enough). Because the remote PCP will be drained later when some pages are allocated/freed on the CPU.
--
Best Regards,
Huang, Ying
Michal Hocko
reviewerQuoted 79 lines · Ying
Quoted 57 lines · Michal Hocko
Quoted 56 lines · Ying
Quoted 40 lines · Michal Hocko
Quoted 23 lines · Ying
Quoted 21 lines · Michal Hocko
Quoted 19 lines · Ying
Hi, Michal,
Quoted 10 lines · Michal Hocko
Quoted 7 lines · Huang Ying
If there is no memory allocation/freeing in the remote pageset after some time (3 seconds for now), the remote pageset will be drained to avoid memory wastage.
But in the current implementation, vmstat updater worker may not be re-queued when we are waiting for the timeout (
pcp->expire!= 0) if there are no vmstat changes, for example, when CPU goes idle.Why is that a problem?
The pages of the remote zone may be kept in the local per-CPU pageset for long time as long as there's no page allocation/freeing on the logical CPU. In addition to the logical CPU goes idle, this is also possible if the logical CPU is busy in the user space.
Quoted 7 lines
Quoted 2 lines
But why is this a problem? Is the scale of the problem sufficient to trigger out of memory situations or be otherwise harmful?
This may trigger premature page reclaiming. The pages in the PCP of the remote zone would have been freed to satisfy the page allocation for the remote zone to avoid page reclaiming. It's highly possible that the local CPU just allocate/free from/to the remote zone temporarily.
I am slightly confused here but I suspect by zone you mean remote pcp. But more importantly is this a concern seen in real workload? Can you quantify it in some manner? E.g. with this patch we have X more kswapd scanning or even hit direct reclaim much less often.
Quoted 3 lines
So,
we should free PCP pages of the remote zone if there is no page allocation/freeing from/to the remote zone for 3 seconds.Quoted 8 lines
Well, I would argue this depends a lot. There are workloads which really like to have CPUs idle and yet they would like to benefit from the allocator fast path after that CPU goes out of idle because idling is their power saving opportunity while workloads want to act quickly after there is something to run.
That being said, we really need some numbers (ideally from real world) that proves this is not just a theoretical concern.
The behavior to drain the PCP of the remote zone (that is, remote PCP) was introduced in commit
4ae7c03943fc("[PATCH] Periodically drain non local pagesets"). The goal of draining was well documented in the change log. IIUC, some of your questions can be answered there?Quoted 4 lines
Quoted 2 lines
This patch just restores the original behavior changed by commit
7cc36bbddde5("vmstat: on-demand vmstat workers V8").Let me repeat. You need some numbers to show this is needed.
I have done some test for this patch as follows,
- Run some workloads, use
numactlto bind CPU to node 0 and memory to node 1. So the PCP of the CPU on node 0 for zone on node 1 will be filled.- After workloads finish, idle for 60s
- Check
/proc/zoneinfoWith the original kernel, the number of pages in the PCP of the CPU on node 0 for zone on node 1 is non-zero after idle. With the patched kernel, that becomes 0 after idle. We avoid to keep pages in the remote PCP during idle.
Quoted 4 lines
This is the number I have. If you think it isn't enough to justify the patch, then I'm OK too (although I think it's enough). Because the remote PCP will be drained later when some pages are allocated/freed on the CPU.
Yes, this doesn't really show any actual correctness problem so I do not think this is sufficient to change the code. You would need to show that the existing behavior is actively harmful.
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 3 lines · Michal Hocko
Yes, this doesn't really show any actual correctness problem so I do not think this is sufficient to change the code. You would need to show that the existing behavior is actively harmful.
Having some pages from a remote NUMA node stuck in a pcp somewhere is making that memory unusable. It is usually rate that these remote pages are needed again and so they may remain there for a long time if the situation is right.
And he is right that the intended behavior of freeing the remote pages has been disabled by the patch.
So I think there is sufficient rationale to apply these fixes.
Hi, Christopher,
Quoted 11 lines · Christopher
Quoted 3 lines · Michal Hocko
Yes, this doesn't really show any actual correctness problem so I do not think this is sufficient to change the code. You would need to show that the existing behavior is actively harmful.
Having some pages from a remote NUMA node stuck in a pcp somewhere is making that memory unusable. It is usually rate that these remote pages are needed again and so they may remain there for a long time if the situation is right.
Quoted 4 lines
And he is right that the intended behavior of freeing the remote pages has been disabled by the patch.
So I think there is sufficient rationale to apply these fixes.
Thanks! Can I get your "Acked-by" or "Reviewed-by" for the patch?
--
Best Regards,
Huang, Ying
Lameter, Christopher
Reviewed-by Christoph LameterQuoted 3 lines · Ying
Quoted 1 line
So I think there is sufficient rationale to apply these fixes.
Thanks! Can I get your "Acked-by" or "Reviewed-by" for the patch?
Quoted 11 lines · Christopher
Quoted 3 lines · Michal Hocko
Yes, this doesn't really show any actual correctness problem so I do not think this is sufficient to change the code. You would need to show that the existing behavior is actively harmful.
Having some pages from a remote NUMA node stuck in a pcp somewhere is making that memory unusable. It is usually rate that these remote pages are needed again and so they may remain there for a long time if the situation is right.
Quoted 4 lines
And he is right that the intended behavior of freeing the remote pages has been disabled by the patch.
So I think there is sufficient rationale to apply these fixes.
I wonder if this the optimum way to handle the NOHZ case? IIUC there we use quiet_vmstat() to call refresh_cpu_vm_stats(). I'd expect if there were pending remote pages to flush, it would be best to do it immediately, and not keep a worker being requeued and only do that after the pcp->expires goes zero.
However quiet_vmstat() even calls the refresh with do_pagesets == false. Why do we even refresh the stats at that moment if the delayed update is pending anyway? And could we maybe make sure that in that case the flush is done on the first delayed update in that case and not expiring like this?
Quoted 23 lines · Vlastimil Babka
Quoted 11 lines · Christopher
Quoted 3 lines · Michal Hocko
Yes, this doesn't really show any actual correctness problem so I do not think this is sufficient to change the code. You would need to show that the existing behavior is actively harmful.
Having some pages from a remote NUMA node stuck in a pcp somewhere is making that memory unusable. It is usually rate that these remote pages are needed again and so they may remain there for a long time if the situation is right.
Quoted 4 lines
And he is right that the intended behavior of freeing the remote pages has been disabled by the patch.
So I think there is sufficient rationale to apply these fixes.
I wonder if this the optimum way to handle the NOHZ case? IIUC there we use
quiet_vmstat()to callrefresh_cpu_vm_stats(). I'd expect if there were pending remote pages to flush, it would be best to do it immediately, and not keep a worker being requeued and only do that after thepcp->expiresgoes zero.
Quoted 3 lines
However
quiet_vmstat()even calls the refresh withdo_pagesets== false. Why do we even refresh the stats at that moment if the delayed update is pending anyway?
According to commit f01f17d3705b ("mm, vmstat: make quiet_vmstat lighter") and the comments in quiet_vmstat(). The pending worker will not be canceled to avoid long latency of idle entry.
Quoted 2 lines
And could we maybe make sure that in that case the flush is done on the first delayed update in that case and not expiring like this?
This sounds reasonable. How to identify whether the current CPU is in NOHZ state? Via tick_get_tick_sched()->tick_stopped?
--
Best Regards,
Huang, Ying