mm/page_alloc: check high-order pages for corruption during PCP operations
Subsystems (2)
PATCH 1/1 mm/page_alloc: check high-order pages for corruption during PCP operations
Eric Dumazet pointed out that commit 44042b449872 ("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists") only checks the head page during PCP refill and allocation operations. This was an oversight and all pages should be checked. This will incur a small performance penalty but it's necessary for correctness.
--- mm/page_alloc.c | 46 +++++++++++++++++++++++----------------------- 1 file changed, 23 insertions(+), 23 deletions(-)
mm/page_alloc.c +23 -23
@@ -2342,23 +2342,36 @@ static inline int check_new_page(struct page *page)return 1 ; } +static bool check_new_pages (struct page *page,unsigned int order) +{ +int i; +for (i =0 ; i < (1 << order); i++) { +struct page *p = page + i; + +if (unlikely (check_new_page (p))) +return true; + } + +return false; +} +#ifdef CONFIG_DEBUG_VM /* * With DEBUG_VM enabled, order-0 pages are checked for expected state when * being allocated from pcp lists. With debug_pagealloc also enabled, they are * also checked when pcp lists are refilled from the free lists. */ -static inline bool check_pcp_refill (struct page *page) +static inline bool check_pcp_refill (struct page *page,unsigned int order) {if (debug_pagealloc_enabled_static ()) -return check_new_page (page); +return check_new_pages (page, order);else return false; } -static inline bool check_new_pcp (struct page *page) +static inline bool check_new_pcp (struct page *page,unsigned int order) { -return check_new_page (page); +return check_new_pages (page, order); }#else /*
@@ -2366,32 +2379,19 @@ static inline bool check_new_pcp(struct page *page)* when pcp lists are being refilled from the free lists. With debug_pagealloc * enabled, they are also checked when being allocated from the pcp lists. */ -static inline bool check_pcp_refill (struct page *page) +static inline bool check_pcp_refill (struct page *page,unsigned int order) { -return check_new_page (page); +return check_new_pages (page, order); } -static inline bool check_new_pcp (struct page *page) +static inline bool check_new_pcp (struct page *page,unsigned int order) {if (debug_pagealloc_enabled_static ()) -return check_new_page (page); +return check_new_pages (page, order);else return false; }#endif /* CONFIG_DEBUG_VM */ -static bool check_new_pages (struct page *page,unsigned int order) -{ -int i; -for (i =0 ; i < (1 << order); i++) { -struct page *p = page + i; - -if (unlikely (check_new_page (p))) -return true; - } - -return false; -} -inline void post_alloc_hook (struct page *page,unsigned int order,gfp_t gfp_flags) {
@@ -3037,7 +3037,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order,if (unlikely (page ==NULL ))break ; -if (unlikely (check_pcp_refill (page))) +if (unlikely (check_pcp_refill (page, order)))continue ;/*
@@ -3641,7 +3641,7 @@ struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, page = list_first_entry(list, struct page, lru);list_del (&page->lru); pcp->count -=1 << order; - }while (check_new_pcp (page)); + }while (check_new_pcp (page, order));return page; }
Quoted 10 lines · Mel Gorman
Eric Dumazet pointed out that commit
44042b449872("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists") only checks the head page during PCP refill and allocation operations. This was an oversight and all pages should be checked. This will incur a small performance penalty but it's necessary for correctness.Fixes 44042b449872 ("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists")Reported-by Eric Dumazet <[email protected]>Signed-off-by Mel Gorman <[email protected]>Acked-by Eric Dumazet <[email protected]>
Quoted 10 lines · Mel Gorman
Eric Dumazet pointed out that commit
44042b449872("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists") only checks the head page during PCP refill and allocation operations. This was an oversight and all pages should be checked. This will incur a small performance penalty but it's necessary for correctness.Fixes 44042b449872 ("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists")Reported-by Eric Dumazet <[email protected]>Signed-off-by Mel Gorman <[email protected]>Acked-by Eric Dumazet <[email protected]>
Quoted 10 lines · Mel Gorman
Eric Dumazet pointed out that commit
44042b449872("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists") only checks the head page during PCP refill and allocation operations. This was an oversight and all pages should be checked. This will incur a small performance penalty but it's necessary for correctness.Fixes 44042b449872 ("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists")Reported-by Eric Dumazet <[email protected]>Signed-off-by Mel Gorman <[email protected]>Acked-by Eric Dumazet <[email protected]>