Follow-up to Leave IRQs enabled for per-cpu page allocations
Subsystems (2)
cover Follow-up to Leave IRQs enabled for per-cpu page allocations
The following two patches are a fixup patch and a simplification as suggested by Vlastimil Babka.
mm/page_alloc.c | 29 +++++++++++------------------ 1 file changed, 11 insertions(+), 18 deletions(-)
Signature · 1 line
2.35.3
PATCH 1/2 mm/page_alloc: Leave IRQs enabled for per-cpu page allocations -fix
As noted by Vlastimil Babka, the migratetype might be wrong if a PCP was not locked so check the migrate type early. Similarly the !pcp check is generally unlikely so explicitly tagging it makes sense.
This is a fix for the mm-unstable patch mm-page_alloc-leave-irqs-enabled-for-per-cpu-page-allocations.patch
--- mm/page_alloc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/page_alloc.c +2 -2
@@ -3516,6 +3516,7 @@ void free_unref_page_list(struct list_head *list)struct zone *zone =page_zone (page);list_del (&page->lru); + migratetype =get_pcppage_migratetype (page);/* Different zone, different pcp lock. */ if (zone != locked_zone) {
@@ -3530,7 +3531,7 @@ void free_unref_page_list(struct list_head *list)*/ pcp_trylock_prepare (UP_flags); pcp =pcp_spin_trylock (zone->per_cpu_pageset); -if (!pcp) { +if (unlikely (!pcp)) {pcp_trylock_finish (UP_flags);free_one_page (zone, page,page_to_pfn (page),0 , migratetype,FPI_NONE );
@@ -3545,7 +3546,6 @@ void free_unref_page_list(struct list_head *list)* Non-isolated types over MIGRATE_PCPTYPES get added * to the MIGRATE_MOVABLE pcp list. */ - migratetype =get_pcppage_migratetype (page);if (unlikely (migratetype >=MIGRATE_PCPTYPES )) migratetype =MIGRATE_MOVABLE ;
Signature · 1 line
2.35.3
PATCH 2/2 mm/page_alloc: Simplify locking during free_unref_page_list
In reply to Mel GormanWhile freeing a large list, the zone lock will be released and reacquired to avoid long hold times since commit c24ad77d962c ("mm/page_alloc.c: avoid excessive IRQ disabled times in free_unref_page_list()"). As suggested by Vlastimil Babka, the lockrelease/reacquire logic can be simplified by reusing the logic that acquires a different lock when changing zones.
--- mm/page_alloc.c | 25 +++++++++---------------- 1 file changed, 9 insertions(+), 16 deletions(-)
mm/page_alloc.c +9 -16
@@ -3518,13 +3518,19 @@ void free_unref_page_list(struct list_head *list)list_del (&page->lru); migratetype =get_pcppage_migratetype (page); -/* Different zone, different pcp lock. */ -if (zone != locked_zone) { +/* +* Either different zone requiring a different pcp lock or +* excessive lock hold times when freeing a large list of +* pages. +*/ +if (zone != locked_zone || batch_count ==SWAP_CLUSTER_MAX ) {if (pcp) {pcp_spin_unlock (pcp);pcp_trylock_finish (UP_flags); } + batch_count =0 ; +/* * trylock is necessary as pages may be getting freed * from IRQ or SoftIRQ context after an IO completion.
@@ -3539,7 +3545,6 @@ void free_unref_page_list(struct list_head *list)continue ; } locked_zone = zone; - batch_count =0 ; }/*
@@ -3551,19 +3556,7 @@ void free_unref_page_list(struct list_head *list)trace_mm_page_free_batched (page);free_unref_page_commit (zone, pcp, page, migratetype,0 ); - -/* -* Guard against excessive lock hold times when freeing -* a large list of pages. Lock will be reacquired if -* necessary on the next iteration. -*/ -if (++batch_count ==SWAP_CLUSTER_MAX ) { -pcp_spin_unlock (pcp); -pcp_trylock_finish (UP_flags); - batch_count =0 ; - pcp =NULL ; - locked_zone =NULL ; - } + batch_count++; }if (pcp) {
Signature · 1 line
2.35.3