mm: drop hotplug lock from lru_add_drain_all
Subsystems (3)
PATCH 1/1 mm: drop hotplug lock from lru_add_drain_all
From: Michal Hocko <[email protected]>
Pulling cpu hotplug locks inside the mm core function like lru_add_drain_all just asks for problems and the recent lockdep splat
[1] just proves this. While the usage in that particular case might be wrong we should prevent from locking as lru_add_drain_all is used at many places. It seems that this is not all that hard to achieve actually.
We have done the same thing for drain_all_pages which is analogous by a459eeb7b852 ("mm, page_alloc: do not depend on cpu hotplug locks inside the allocator"). All we have to care about is to handle
- the work item might be executed on a different cpu in worker from unbound pool so it doesn't run on pinned on the cpu
- we have to make sure that we do not race with page_alloc_cpu_dead calling lru_add_drain_cpu
the first part is already handled because the worker calls lru_add_drain which disables preemption when calling lru_add_drain_cpu on the local cpu it is draining. The later is true because page_alloc_cpu_dead is called on the controlling CPU after the hotplugged CPU vanished completely.
[1] http://lkml.kernel.org/r/[email protected]
--- include/linux/swap.h | 1 - mm/memory_hotplug.c | 2 +- mm/swap.c | 16 ++++++++-------- 3 files changed, 9 insertions(+), 10 deletions(-)
include/linux/swap.h +0 -1
@@ -331,7 +331,6 @@ extern void mark_page_accessed(struct page *);extern void lru_add_drain (void );extern void lru_add_drain_cpu (int cpu);extern void lru_add_drain_all (void ); -extern void lru_add_drain_all_cpuslocked (void );extern void rotate_reclaimable_page (struct page *page);extern void deactivate_file_page (struct page *page);extern void mark_page_lazyfree (struct page *page);
mm/memory_hotplug.c +1 -1
@@ -1641,7 +1641,7 @@ static int __ref __offline_pages(unsigned long start_pfn, goto failed_removal;cond_resched (); -lru_add_drain_all_cpuslocked (); +lru_add_drain_all ();drain_all_pages (zone); pfn =scan_movable_pages (start_pfn, end_pfn);
mm/swap.c +8 -8
@@ -688,7 +688,14 @@ static void lru_add_drain_per_cpu(struct work_struct *dummy) static DEFINE_PER_CPU(struct work_struct, lru_add_drain_work); -void lru_add_drain_all_cpuslocked (void ) +/* +* Doesn't need any cpu hotplug locking because we do rely on per-cpu +* kworkers being shut down before our page_alloc_cpu_dead callback is +* executed on the offlined cpu. +* Calling this function with cpu hotplug locks held can actually lead +* to obscure indirect dependencies via WQ context. +*/ +void lru_add_drain_all (void ) {static DEFINE_MUTEX (lock);static struct cpumask has_work;
@@ -724,13 +731,6 @@ void lru_add_drain_all_cpuslocked(void)mutex_unlock (&lock); } -void lru_add_drain_all (void ) -{ -get_online_cpus (); -lru_add_drain_all_cpuslocked (); -put_online_cpus (); -} -/** * release_pages - batched put_page() * @pages: array of pages to release
Signature · 7 lines
2.15.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>