mm: move pcp and lru-pcp drainging into vmstat_wq
Subsystems (3)
RFC PATCH 1/1 mm: move pcp and lru-pcp drainging into vmstat_wq
From: Michal Hocko <[email protected]>
We currently have 2 specific WQ_RECLAIM workqueues. One for updating pcp stats vmstat_wq and one dedicated to drain per cpu lru caches. This seems more than necessary because both can run on a single WQ. Both do not block on locks requiring a memory allocation nor perform any allocations themselves. We will save one rescuer thread this way.
On the other hand drain_all_pages queues work on the system wq which doesn't have rescuer and so this depend on memory allocation (when all workers are stuck allocating and new ones cannot be created). This is not critical as there should be somebody invoking the OOM killer (e.g. the forking worker) and get the situation unstuck and eventually performs the draining. Quite annoying though. This worker should be using WQ_RECLAIM as well. We can reuse the same one as for lru draining and vmstat.
---
Hi,
Tetsuo has noted that drain_all_pages doesn't use WQ_RECLAIM [1] and asked whether we can move the worker to the vmstat_wq which is WQ_RECLAIM. I think the deadlock he has described shouldn't happen but it would be really better to have the rescuer. I also think that we do not really need 2 or more workqueues and also pull lru draining in.
What do you think? Please note I haven't tested it yet.
[1] http://lkml.kernel.org/r/[email protected]
mm/internal.h | 6 ++++++ mm/page_alloc.c | 2 +- mm/swap.c | 20 +------------------- mm/vmstat.c | 11 ++++++----- 4 files changed, 14 insertions(+), 25 deletions(-)
mm/internal.h +6 -0
@@ -498,4 +498,10 @@ extern const struct trace_print_flags pageflag_names[];extern const struct trace_print_flags vmaflag_names[];extern const struct trace_print_flags gfpflag_names[]; +/* +* only for MM internal work items which do not depend on +* any allocations or locks which might depend on allocations +*/ +extern struct workqueue_struct *vmstat_wq; +#endif /* __MM_INTERNAL_H */
mm/page_alloc.c +1 -1
@@ -2419,7 +2419,7 @@ void drain_all_pages(struct zone *zone)for_each_cpu (cpu, &cpus_with_pcps) {struct work_struct *work =per_cpu_ptr (&pcpu_drain, cpu);INIT_WORK (work, drain_local_pages_wq); -schedule_work_on (cpu, work); +queue_work_on (cpu, vmstat_wq, work); }for_each_cpu (cpu, &cpus_with_pcps)flush_work (per_cpu_ptr (&pcpu_drain, cpu));
mm/swap.c +1 -19
@@ -670,24 +670,6 @@ static void lru_add_drain_per_cpu(struct work_struct *dummy) static DEFINE_PER_CPU(struct work_struct, lru_add_drain_work); -/* -* lru_add_drain_wq is used to do lru_add_drain_all() from a WQ_MEM_RECLAIM -* workqueue, aiding in getting memory freed. -*/ -static struct workqueue_struct *lru_add_drain_wq; - -static int __init lru_init (void ) -{ - lru_add_drain_wq =alloc_workqueue ("lru-add-drain" ,WQ_MEM_RECLAIM ,0 ); - -if (WARN (!lru_add_drain_wq, -"Failed to create workqueue lru_add_drain_wq" )) -return -ENOMEM ; - -return 0 ; -} -early_initcall (lru_init); -void lru_add_drain_all (void ) {static DEFINE_MUTEX (lock);
@@ -707,7 +689,7 @@ void lru_add_drain_all(void) pagevec_count(&per_cpu(lru_deactivate_pvecs, cpu)) || need_activate_page_drain(cpu)) {INIT_WORK (work, lru_add_drain_per_cpu); -queue_work_on (cpu, lru_add_drain_wq, work); +queue_work_on (cpu, vmstat_wq, work);cpumask_set_cpu (cpu, &has_work); } }
mm/vmstat.c +6 -5
@@ -1548,8 +1548,8 @@ static const struct file_operations proc_vmstat_file_operations = { };#endif /* CONFIG_PROC_FS */ +struct workqueue_struct *vmstat_wq;#ifdef CONFIG_SMP -static struct workqueue_struct *vmstat_wq; static DEFINE_PER_CPU(struct delayed_work, vmstat_work);int sysctl_stat_interval__read_mostly =HZ ;
@@ -1715,7 +1715,6 @@ static void __init start_shepherd_timer(void)INIT_DEFERRABLE_WORK (per_cpu_ptr (&vmstat_work, cpu), vmstat_update); - vmstat_wq =alloc_workqueue ("vmstat" ,WQ_FREEZABLE |WQ_MEM_RECLAIM ,0 );schedule_delayed_work (&shepherd,round_jiffies_relative (sysctl_stat_interval)); }
@@ -1763,9 +1762,11 @@ static int vmstat_cpu_dead(unsigned int cpu)static int __init setup_vmstat (void ) { -#ifdef CONFIG_SMP -int ret; +int ret =0 ; + + vmstat_wq =alloc_workqueue ("vmstat" ,WQ_FREEZABLE |WQ_MEM_RECLAIM ,0 ); +#ifdef CONFIG_SMP ret =cpuhp_setup_state_nocalls (CPUHP_MM_VMSTAT_DEAD ,"mm/vmstat:dead" ,NULL , vmstat_cpu_dead);if (ret <0 )
@@ -1789,7 +1790,7 @@ static int __init setup_vmstat(void)proc_create ("vmstat" ,S_IRUGO ,NULL , &proc_vmstat_file_operations);proc_create ("zoneinfo" ,S_IRUGO ,NULL , &proc_zoneinfo_file_operations);#endif -return 0 ; +return ret; }module_init (setup_vmstat)
Signature · 7 lines
2.11.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 21 lines · Michal Hocko
From: Michal Hocko <[email protected]>
We currently have 2 specific
WQ_RECLAIMworkqueues. One for updating pcp statsvmstat_wqand one dedicated to drain per cpu lru caches. This seems more than necessary because both can run on a single WQ. Both do not block on locks requiring a memory allocation nor perform any allocations themselves. We will save one rescuer thread this way.On the other hand
drain_all_pagesqueues work on the system wq which doesn't have rescuer and so this depend on memory allocation (when all workers are stuck allocating and new ones cannot be created). This is not critical as there should be somebody invoking the OOM killer (e.g. the forking worker) and get the situation unstuck and eventually performs the draining. Quite annoying though. This worker should be usingWQ_RECLAIMas well. We can reuse the same one as for lru draining and vmstat.Suggested-by Tetsuo Handa <[email protected]>Signed-off-by Michal Hocko <[email protected]>---
Quoted 8 lines
Hi,
Tetsuo has noted thatdrain_all_pagesdoesn't useWQ_RECLAIM[1] and asked whether we can move the worker to thevmstat_wqwhich isWQ_RECLAIM. I think the deadlock he has described shouldn't happen but it would be really better to have the rescuer. I also think that we do not really need 2 or more workqueues and also pull lru draining in.What do you think? Please note I haven't tested it yet.
Why not, I guess, of course I may be overlooking some subtlety. You could have CC'd Tejun and Christoph. Watch out for the init order though, maybe? Is there no caller of the lru/pcp drain before module_init(setup_vmstat) happens?
Also one nit below.
Quoted 97 lines
[1] http://lkml.kernel.org/r/[email protected]
mm/internal.h | 6 ++++++ mm/page_alloc.c | 2 +- mm/swap.c | 20 +------------------- mm/vmstat.c | 11 ++++++----- 4 files changed, 14 insertions(+), 25 deletions(-)mm/internal.h +6
-0@@ -498,4 +498,10 @@ extern const struct trace_print_flags pageflag_names[];extern const struct trace_print_flags vmaflag_names[];extern const struct trace_print_flags gfpflag_names[]; +/* +* only for MM internal work items which do not depend on +* any allocations or locks which might depend on allocations +*/ +extern struct workqueue_struct *vmstat_wq; +#endif /* __MM_INTERNAL_H */ mm/page_alloc.c +1
-1@@ -2419,7 +2419,7 @@ void drain_all_pages(struct zone *zone)for_each_cpu (cpu, &cpus_with_pcps) {struct work_struct *work =per_cpu_ptr (&pcpu_drain, cpu);INIT_WORK (work, drain_local_pages_wq); -schedule_work_on (cpu, work); +queue_work_on (cpu, vmstat_wq, work); }for_each_cpu (cpu, &cpus_with_pcps)flush_work (per_cpu_ptr (&pcpu_drain, cpu));mm/swap.c +1
-19@@ -670,24 +670,6 @@ static void lru_add_drain_per_cpu(struct work_struct *dummy) static DEFINE_PER_CPU(struct work_struct, lru_add_drain_work); -/* -* lru_add_drain_wq is used to do lru_add_drain_all() from a WQ_MEM_RECLAIM -* workqueue, aiding in getting memory freed. -*/ -static struct workqueue_struct *lru_add_drain_wq; - -static int __init lru_init (void ) -{ - lru_add_drain_wq =alloc_workqueue ("lru-add-drain" ,WQ_MEM_RECLAIM ,0 ); - -if (WARN (!lru_add_drain_wq, -"Failed to create workqueue lru_add_drain_wq" )) -return -ENOMEM ; - -return 0 ; -} -early_initcall (lru_init); -void lru_add_drain_all (void ) {static DEFINE_MUTEX (lock);@@ -707,7 +689,7 @@ void lru_add_drain_all(void) pagevec_count(&per_cpu(lru_deactivate_pvecs, cpu)) || need_activate_page_drain(cpu)) {INIT_WORK (work, lru_add_drain_per_cpu); -queue_work_on (cpu, lru_add_drain_wq, work); +queue_work_on (cpu, vmstat_wq, work);cpumask_set_cpu (cpu, &has_work); } }mm/vmstat.c +1
-2@@ -1548,8 +1548,8 @@ static const struct file_operations proc_vmstat_file_operations = { };#endif /* CONFIG_PROC_FS */ +struct workqueue_struct *vmstat_wq;#ifdef CONFIG_SMP -static struct workqueue_struct *vmstat_wq; static DEFINE_PER_CPU(struct delayed_work, vmstat_work);int sysctl_stat_interval__read_mostly =HZ ;@@ -1715,7 +1715,6 @@ static void __init start_shepherd_timer(void)INIT_DEFERRABLE_WORK (per_cpu_ptr (&vmstat_work, cpu), vmstat_update); - vmstat_wq =alloc_workqueue ("vmstat" ,WQ_FREEZABLE |WQ_MEM_RECLAIM ,0 );schedule_delayed_work (&shepherd,round_jiffies_relative (sysctl_stat_interval)); }
mm/vmstat.c static int vmstat_cpu_dead(unsigned int cpu) · line 1767
Quoted 8 lines
mm/vmstat.c +3
-2@@ -1763,9 +1762,11 @@ static int vmstat_cpu_dead(unsigned int cpu)static int __init setup_vmstat (void ) { -#ifdef CONFIG_SMP -int ret; +int ret =0 ; + + vmstat_wq =alloc_workqueue ("vmstat" ,WQ_FREEZABLE |WQ_MEM_RECLAIM ,0 );
Did you want to set ret to -ENOMEM if the alloc fails, or something? Otherwise I don't see why the changes.
Quoted 7 lines
mm/vmstat.c +1
-0+#ifdef CONFIG_SMP ret =cpuhp_setup_state_nocalls (CPUHP_MM_VMSTAT_DEAD ,"mm/vmstat:dead" ,NULL , vmstat_cpu_dead);if (ret <0 )
mm/vmstat.c static int __init setup_vmstat(void) · line 1796
Quoted 8 lines
mm/vmstat.c +1
-1@@ -1789,7 +1790,7 @@ static int __init setup_vmstat(void)proc_create ("vmstat" ,S_IRUGO ,NULL , &proc_vmstat_file_operations);proc_create ("zoneinfo" ,S_IRUGO ,NULL , &proc_zoneinfo_file_operations);#endif -return 0 ; +return ret; }module_init (setup_vmstat)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 29 lines · Vlastimil Babka
Quoted 28 lines · Michal Hocko
From: Michal Hocko <[email protected]>
We currently have 2 specific
WQ_RECLAIMworkqueues. One for updating pcp statsvmstat_wqand one dedicated to drain per cpu lru caches. This seems more than necessary because both can run on a single WQ. Both do not block on locks requiring a memory allocation nor perform any allocations themselves. We will save one rescuer thread this way.On the other hand
drain_all_pagesqueues work on the system wq which doesn't have rescuer and so this depend on memory allocation (when all workers are stuck allocating and new ones cannot be created). This is not critical as there should be somebody invoking the OOM killer (e.g. the forking worker) and get the situation unstuck and eventually performs the draining. Quite annoying though. This worker should be usingWQ_RECLAIMas well. We can reuse the same one as for lru draining and vmstat.Suggested-by Tetsuo Handa <[email protected]>Signed-off-by Michal Hocko <[email protected]>---
Hi,
Tetsuo has noted thatdrain_all_pagesdoesn't useWQ_RECLAIM[1] and asked whether we can move the worker to thevmstat_wqwhich isWQ_RECLAIM. I think the deadlock he has described shouldn't happen but it would be really better to have the rescuer. I also think that we do not really need 2 or more workqueues and also pull lru draining in.
Quoted 4 lines
Quoted 1 line
What do you think? Please note I haven't tested it yet.
Why not, I guess, of course I may be overlooking some subtlety. You could have CC'd Tejun and Christoph.
will do on the next submission
Quoted 2 lines
Watch out for the init order though, maybe? Is there no caller of the lru/pcp drain before
module_init(setup_vmstat)happens?
Hard to tell. I expect that there shouldn't be but I will add
mm/page_alloc.c +7 -0
@@ -2370,6 +2370,13 @@ void drain_all_pages(struct zone *zone)*/ static cpumask_t cpus_with_pcps; +/* +* Make sure nobody triggers this path before vmstat_wq is fully +* initialized. +*/ +if (WARN_ON (!vmstat_wq)) +return ; +/* Workqueues cannot recurse */ if (current->flags &PF_WQ_WORKER )return ;
mm/swap.c +7 -0
@@ -676,6 +676,13 @@ void lru_add_drain_all(void)static struct cpumask has_work;int cpu; +/* +* Make sure nobody triggers this path before vmstat_wq is fully +* initialized. +*/ +if (WARN_ON (!vmstat_wq)) +return ; +mutex_lock (&lock);get_online_cpus ();cpumask_clear (&has_work);
to be sure.
[...]
Quoted 12 lines
Review static int vmstat_cpu_dead(unsigned int cpu) · line 1767
Quoted 9 lines
(quoted diff) +3
-2@@ -1763,9 +1762,11 @@ static int vmstat_cpu_dead(unsigned int cpu) static int __init setup_vmstat(void) { -#ifdef CONFIG_SMP - int ret; + int ret = 0; + + vmstat_wq = alloc_workqueue("vmstat", WQ_FREEZABLE|WQ_MEM_RECLAIM, 0);Did you want to set ret to
-ENOMEMif the alloc fails, or something? Otherwise I don't see why the changes.
no I just didn't want to get defined but not used warning for CONFIG_SMP=n
Quoted 16 lines
Quoted 16 lines
(quoted diff) +2
-1+#ifdef CONFIG_SMP ret = cpuhp_setup_state_nocalls(CPUHP_MM_VMSTAT_DEAD, "mm/vmstat:dead", NULL, vmstat_cpu_dead); if (ret < 0)@@ -1789,7 +1790,7 @@ static int __init setup_vmstat(void) proc_create("vmstat", S_IRUGO, NULL, &proc_vmstat_file_operations); proc_create("zoneinfo", S_IRUGO, NULL, &proc_zoneinfo_file_operations); #endif - return 0; + return ret; } module_init(setup_vmstat)
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · Michal Hocko
From: Michal Hocko <[email protected]>
We currently have 2 specific
WQ_RECLAIMworkqueues. One for updating pcp statsvmstat_wqand one dedicated to drain per cpu lru caches. This seems more than necessary because both can run on a single WQ. Both do not block on locks requiring a memory allocation nor perform any allocations themselves. We will save one rescuer thread this way.
True.
Quoted 9 lines
On the other hand
drain_all_pagesqueues work on the system wq which doesn't have rescuer and so this depend on memory allocation (when all workers are stuck allocating and new ones cannot be created). This is not critical as there should be somebody invoking the OOM killer (e.g. the forking worker) and get the situation unstuck and eventually performs the draining. Quite annoying though. This worker should be usingWQ_RECLAIMas well. We can reuse the same one as for lru draining and vmstat.
That was still debatable which is why I didn't go that route. The drain itself is unlikely to fix anything with the possible exception of high-order pages. There are just too many reasons why direct reclaim can return 0 reclaimed pages making the drain is redundant but I couldn't decide what a better alternative would be and more importantly, how to measure it. The fact it allocates in that path is currently unfortunate but I couldn't convince myself it deserved a dedicated rescuer.
I don't object to it being actually moved. I have a slight concern that it could somehow starve a vmstat update while frequent drains happen during reclaim though which potentially compounds the problem. It could be offset by a variety of other factors but if it ever is an issue, it'll show up and the paths that really matter check the vmstats directly instead of waiting for an update.
Quoted 4 lines
Suggested-by Tetsuo Handa <[email protected]>Signed-off-by Michal Hocko <[email protected]>---
Quoted 9 lines
Hi,
Tetsuo has noted thatdrain_all_pagesdoesn't useWQ_RECLAIM[1] and asked whether we can move the worker to thevmstat_wqwhich isWQ_RECLAIM. I think the deadlock he has described shouldn't happen but it would be really better to have the rescuer. I also think that we do not really need 2 or more workqueues and also pull lru draining in.What do you think? Please note I haven't tested it yet.
As an aside, the LRU drain could also avoid a get_online_cpus() which is surprisingly heavy handed for an operation that can happen quite frequently during compaction or migration. Maybe not enough to make a big deal of but it's relatively low hanging fruit.
The altering of the return value in setup_vmstat was mildly surprising as it increases the severity of registering the vmstat callback for memory hotplug so maybe split that out and appears unrelated.
It also feels like vmstat is now a misleading name for something that handles vmstat, lru drains and per-cpu drains but that's cosmetic.
Fundamentally I have nothing against the patch.
Signature · 8 lines
Mel Gorman SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 22 lines · Mel Gorman
Quoted 8 lines · Michal Hocko
From: Michal Hocko <[email protected]>
We currently have 2 specific
WQ_RECLAIMworkqueues. One for updating pcp statsvmstat_wqand one dedicated to drain per cpu lru caches. This seems more than necessary because both can run on a single WQ. Both do not block on locks requiring a memory allocation nor perform any allocations themselves. We will save one rescuer thread this way.True.
Quoted 9 lines
On the other hand
drain_all_pagesqueues work on the system wq which doesn't have rescuer and so this depend on memory allocation (when all workers are stuck allocating and new ones cannot be created). This is not critical as there should be somebody invoking the OOM killer (e.g. the forking worker) and get the situation unstuck and eventually performs the draining. Quite annoying though. This worker should be usingWQ_RECLAIMas well. We can reuse the same one as for lru draining and vmstat.
Quoted 7 lines
That was still debatable which is why I didn't go that route. The drain itself is unlikely to fix anything with the possible exception of high-order pages. There are just too many reasons why direct reclaim can return 0 reclaimed pages making the drain is redundant but I couldn't decide what a better alternative would be and more importantly, how to measure it. The fact it allocates in that path is currently unfortunate but I couldn't convince myself it deserved a dedicated rescuer.
agreed
Quoted 6 lines
I don't object to it being actually moved. I have a slight concern that it could somehow starve a vmstat update while frequent drains happen during reclaim though which potentially compounds the problem. It could be offset by a variety of other factors but if it ever is an issue, it'll show up and the paths that really matter check the vmstats directly instead of waiting for an update.
vmstat updates can tolared delays, that's we we are using deferable scheduling in the first place so I am not really worried about that. Any user which needs a better precision should use *_snapshot API.
Quoted 11 lines
Quoted 11 lines
Suggested-by Tetsuo Handa <[email protected]>Signed-off-by Michal Hocko <[email protected]>---
Hi,
Tetsuo has noted thatdrain_all_pagesdoesn't useWQ_RECLAIM[1] and asked whether we can move the worker to thevmstat_wqwhich isWQ_RECLAIM. I think the deadlock he has described shouldn't happen but it would be really better to have the rescuer. I also think that we do not really need 2 or more workqueues and also pull lru draining in.
Quoted 7 lines
Quoted 2 lines
What do you think? Please note I haven't tested it yet.
As an aside, the LRU drain could also avoid a
get_online_cpus()which is surprisingly heavy handed for an operation that can happen quite frequently during compaction or migration. Maybe not enough to make a big deal of but it's relatively low hanging fruit.
Yeah, this is sitting on my todo list already, I just didn't give it a priority.
Quoted 3 lines
The altering of the return value in
setup_vmstatwas mildly surprising as it increases the severity of registering the vmstat callback for memory hotplug so maybe split that out and appears unrelated.
not sure I understand. What do you mean?
Quoted 2 lines
It also feels like vmstat is now a misleading name for something that handles vmstat, lru drains and per-cpu drains but that's cosmetic.
yeah a better name sounds like a good thing. mm_nonblock_wq?
Quoted 1 line
Fundamentally I have nothing against the patch.
Thanks for the review. I will sit on it and give it some testing to see how it behaves and post after I get back from vacation (after 20th)
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 11 lines · Michal Hocko
Quoted 6 lines
I don't object to it being actually moved. I have a slight concern that it could somehow starve a vmstat update while frequent drains happen during reclaim though which potentially compounds the problem. It could be offset by a variety of other factors but if it ever is an issue, it'll show up and the paths that really matter check the vmstats directly instead of waiting for an update.
vmstat updates can tolared delays, that's we we are using deferable scheduling in the first place so I am not really worried about that. Any user which needs a better precision should use
*_snapshotAPI.
Agreed, we already had cases where deferred vmstat updates had problems and were resolved by using _snapshot. It's a slight concern only and I'd be surprised if the _snapshot usage didn't cover it.
Quoted 6 lines
Quoted 3 lines
The altering of the return value in
setup_vmstatwas mildly surprising as it increases the severity of registering the vmstat callback for memory hotplug so maybe split that out and appears unrelated.not sure I understand. What do you mean?
This hunk
@@ -1763,9 +1762,11 @@ static int vmstat_cpu_dead(unsigned int cpu)
static int __init setup_vmstat(void)
{
-#ifdef CONFIG_SMP
- int ret;
+ int ret = 0;
+
+ vmstat_wq = alloc_workqueue("vmstat", WQ_FREEZABLE|WQ_MEM_RECLAIM, 0);
+#ifdef CONFIG_SMP
ret = cpuhp_setup_state_nocalls(CPUHP_MM_VMSTAT_DEAD, "mm/vmstat:dead",
NULL, vmstat_cpu_dead);
if (ret < 0)
@@ -1789,7 +1790,7 @@ static int __init setup_vmstat(void)
proc_create("vmstat", S_IRUGO, NULL, &proc_vmstat_file_operations);
proc_create("zoneinfo", S_IRUGO, NULL, &proc_zoneinfo_file_operations); #endif
- return 0;
+ return ret;
A failed register of vmstat_cpu_dead is returning the failure code in an init function now. Chances are it'll never hit but it didn't seem related to the patches general intent.
Quoted 5 lines
Quoted 2 lines
It also feels like vmstat is now a misleading name for something that handles vmstat, lru drains and per-cpu drains but that's cosmetic.
yeah a better name sounds like a good thing.
mm_nonblock_wq?
it's not always non-blocking. Maybe mm_percpu_wq to describev a workqueue that handles a variety of MM-related per-cpu updates?
Signature · 8 lines
Mel Gorman SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Mel Gorman
Quoted 5 lines
Quoted 2 lines
It also feels like vmstat is now a misleading name for something that handles vmstat, lru drains and per-cpu drains but that's cosmetic.
yeah a better name sounds like a good thing.
mm_nonblock_wq?it's not always non-blocking. Maybe
mm_percpu_wqto describev a workqueue that handles a variety of MM-related per-cpu updates?
Why not make it global like ones created by workqueue_init_early() ?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines · Tetsuo Handa
Quoted 3 lines · Mel Gorman
Quoted 3 lines
Quoted 2 lines
It also feels like vmstat is now a misleading name for something that handles vmstat, lru drains and per-cpu drains but that's cosmetic.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
yeah a better name sounds like a good thing.
mm_nonblock_wq?it's not always non-blocking. Maybe
mm_percpu_wqto describev a workqueue that handles a variety of MM-related per-cpu updates?Why not make it global like ones created by
workqueue_init_early()?
I can see alloc_workqueue_attrs in that path so we can hit the page allocator and if unlucky try to drain_all_pages. We might have more even before this. So I think we still need a check for the WQ being initialized already. I do not have a strong preference to when to allocate it. Moving the initialization out to workqueue_init_early wold mean that the WQ would have to be visible outside of the mm proper which is not ideal. We can live with that, though, so I will move it there if this is a prevalent opinion.
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 48 lines · Mel Gorman
Quoted 11 lines · Michal Hocko
Quoted 6 lines
I don't object to it being actually moved. I have a slight concern that it could somehow starve a vmstat update while frequent drains happen during reclaim though which potentially compounds the problem. It could be offset by a variety of other factors but if it ever is an issue, it'll show up and the paths that really matter check the vmstats directly instead of waiting for an update.
vmstat updates can tolared delays, that's we we are using deferable scheduling in the first place so I am not really worried about that. Any user which needs a better precision should use
*_snapshotAPI.Agreed, we already had cases where deferred vmstat updates had problems and were resolved by using
_snapshot. It's a slight concern only and I'd be surprised if the_snapshotusage didn't cover it.Quoted 6 lines
Quoted 3 lines
The altering of the return value in
setup_vmstatwas mildly surprising as it increases the severity of registering the vmstat callback for memory hotplug so maybe split that out and appears unrelated.not sure I understand. What do you mean?
This hunk
mm/vmstat.c +5
-3@@ -1763,9 +1762,11 @@ static int vmstat_cpu_dead(unsigned int cpu)static int __init setup_vmstat (void ) { -#ifdef CONFIG_SMP -int ret; +int ret =0 ; + + vmstat_wq =alloc_workqueue ("vmstat" ,WQ_FREEZABLE |WQ_MEM_RECLAIM ,0 ); +#ifdef CONFIG_SMP ret =cpuhp_setup_state_nocalls (CPUHP_MM_VMSTAT_DEAD ,"mm/vmstat:dead" ,NULL , vmstat_cpu_dead);if (ret <0 )@@ -1789,7 +1790,7 @@ static int __init setup_vmstat(void)proc_create ("vmstat" ,S_IRUGO ,NULL , &proc_vmstat_file_operations);proc_create ("zoneinfo" ,S_IRUGO ,NULL , &proc_zoneinfo_file_operations);#endif -return 0 ; +return ret;
Quoted 3 lines
A failed register of
vmstat_cpu_deadis returning the failure code in an init function now. Chances are it'll never hit but it didn't seem related to the patches general intent.
Ohh, I see now. I will keep the original behavior.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
It also feels like vmstat is now a misleading name for something that handles vmstat, lru drains and per-cpu drains but that's cosmetic.
yeah a better name sounds like a good thing.
mm_nonblock_wq?it's not always non-blocking. Maybe
mm_percpu_wqto describev a workqueue that handles a variety of MM-related per-cpu updates?
Why not, I do not have a strong preference. The WQ is already documented for its requirements on workers so the name doesn't really have to be explicit about blocking on allocations.
Thanks!
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>