mm/memory_hotplug: Don't take the cpu_hotplug_lock
Subsystems (2)
RFC PATCH 1/1 mm/memory_hotplug: Don't take the cpu_hotplug_lock
Commit 9852a7212324 ("mm: drop hotplug lock from lru_add_drain_all()") states that lru_add_drain_all() "Doesn't need any cpu hotplug locking because we do rely on per-cpu kworkers being shut down before our page_alloc_cpu_dead callback is executed on the offlined cpu."
And also "Calling this function with cpu hotplug locks held can actually lead to obscure indirect dependencies via WQ context.".
Since commit 3f906ba23689 ("mm/memory-hotplug: switch locking to a percpu rwsem") we do a cpus_read_lock() in mem_hotplug_begin().
I don't see how that lock is still helpful, we already hold the device_hotplug_lock to protect try_offline_node(), which is AFAIK one problematic part that can race with CPU hotplug. If it is still necessary, we should document why.
--- mm/memory_hotplug.c | 2 -- 1 file changed, 2 deletions(-)
mm/memory_hotplug.c +0 -2
@@ -86,14 +86,12 @@ __setup("memhp_default_state=", setup_memhp_default_state);void mem_hotplug_begin (void ) { -cpus_read_lock ();percpu_down_write (&mem_hotplug_lock); }void mem_hotplug_done (void ) {percpu_up_write (&mem_hotplug_lock); -cpus_read_unlock (); }u64 max_mem_size =U64_MAX ;
Signature · 1 line
2.21.0
Quoted 8 lines · David Hildenbrand
Commit
9852a7212324("mm: drop hotplug lock fromlru_add_drain_all()") states thatlru_add_drain_all()"Doesn't need any cpu hotplug locking because we do rely on per-cpu kworkers being shut down before ourpage_alloc_cpu_deadcallback is executed on the offlined cpu."And also "Calling this function with cpu hotplug locks held can actually lead to obscure indirect dependencies via WQ context.".
Quoted 7 lines
Since commit
3f906ba23689("mm/memory-hotplug: switch locking to a percpu rwsem") we do acpus_read_lock()inmem_hotplug_begin().I don't see how that lock is still helpful, we already hold the
device_hotplug_lockto protecttry_offline_node(), which is AFAIK one problematic part that can race with CPU hotplug. If it is still necessary, we should document why.
I have forgot all the juicy details. Maybe Thomas remembers. The previous recursive home grown locking was just terrible. I do not see stop_machine being used in the memory hotplug anymore.
I do support this kind of removal because binding CPU and MEM hotplug locks is fragile and wrong. But this patch really needs more explanation on why this is safe. In other words what does cpu_read_lock protects from in mem hotplug paths.
Quoted 32 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Thomas Gleixner <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- mm/memory_hotplug.c | 2 -- 1 file changed, 2 deletions(-)mm/memory_hotplug.c +0
-2@@ -86,14 +86,12 @@ __setup("memhp_default_state=", setup_memhp_default_state);void mem_hotplug_begin (void ) { -cpus_read_lock ();percpu_down_write (&mem_hotplug_lock); }void mem_hotplug_done (void ) {percpu_up_write (&mem_hotplug_lock); -cpus_read_unlock (); }u64 max_mem_size =U64_MAX ;Signature · 1 line
2.21.0
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 17 lines · Michal Hocko
Quoted 8 lines · David Hildenbrand
Commit
9852a7212324("mm: drop hotplug lock fromlru_add_drain_all()") states thatlru_add_drain_all()"Doesn't need any cpu hotplug locking because we do rely on per-cpu kworkers being shut down before ourpage_alloc_cpu_deadcallback is executed on the offlined cpu."And also "Calling this function with cpu hotplug locks held can actually lead to obscure indirect dependencies via WQ context.".
Quoted 7 lines
Since commit
3f906ba23689("mm/memory-hotplug: switch locking to a percpu rwsem") we do acpus_read_lock()inmem_hotplug_begin().I don't see how that lock is still helpful, we already hold the
device_hotplug_lockto protecttry_offline_node(), which is AFAIK one problematic part that can race with CPU hotplug. If it is still necessary, we should document why.
Quoted 8 lines
I have forgot all the juicy details. Maybe Thomas remembers. The previous recursive home grown locking was just terrible. I do not see
stop_machinebeing used in the memory hotplug anymore.I do support this kind of removal because binding CPU and MEM hotplug locks is fragile and wrong. But this patch really needs more explanation on why this is safe. In other words what does
cpu_read_lockprotects from in mem hotplug paths.
And that is the purpose of marking this RFC, because I am not aware of any :) Hopefully Thomas can clarify if we are missing something important (undocumented) here - if so I'll document it.
Signature · 3 lines
Thanks, David / dhildenb