FAILED: patch "[PATCH] mm: hugetlb: fix a race between freeing and dissolving the" failed to apply to 4.4-stable tree
Subsystems (3)
[email protected]
maintainerThe patch below does not apply to the 4.4-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.
thanks,
greg k-h
------------------ original commit in Linus's tree ------------------
From 7ffddd499ba6122b1a07828f023d1d67629aa017 Mon Sep 17 00:00:00 2001
From: Muchun Song <[email protected]>
Date: Thu, 4 Feb 2021 18:32:06 -0800
Subject: [PATCH] mm: hugetlb: fix a race between freeing and dissolving the
page
There is a race condition between __free_huge_page() and dissolve_free_huge_page().
CPU0: CPU1:
// page_count(page) == 1
put_page(page)
__free_huge_page(page)
dissolve_free_huge_page(page)
spin_lock(&hugetlb_lock)
// PageHuge(page) && !page_count(page)
update_and_free_page(page)
// page is freed to the buddy
spin_unlock(&hugetlb_lock)
spin_lock(&hugetlb_lock)
clear_page_huge_active(page)
enqueue_huge_page(page)
// It is wrong, the page is already freed
spin_unlock(&hugetlb_lock)The race window is between put_page() and dissolve_free_huge_page().
We should make sure that the page is already on the free list when it is dissolved.
As a result __free_huge_page would corrupt page(s) already in the buddy allocator.
mm/hugetlb.c +39 -0
@@ -79,6 +79,21 @@ DEFINE_SPINLOCK(hugetlb_lock);static int num_fault_mutexes;struct mutex *hugetlb_fault_mutex_table____cacheline_aligned_in_smp ; +static inline bool PageHugeFreed (struct page *head) +{ +return page_private (head +4 ) ==-1UL ; +} + +static inline void SetPageHugeFreed (struct page *head) +{ +set_page_private (head +4 ,-1UL ); +} + +static inline void ClearPageHugeFreed (struct page *head) +{ +set_page_private (head +4 ,0 ); +} +/* Forward declaration */ static int hugetlb_acct_memory (struct hstate *h,long delta);
@@ -1028,6 +1043,7 @@ static void enqueue_huge_page(struct hstate *h, struct page *page)list_move (&page->lru, &h->hugepage_freelists[nid]); h->free_huge_pages++; h->free_huge_pages_node[nid]++; +SetPageHugeFreed (page); }static struct page *dequeue_huge_page_node_exact (struct hstate *h,int nid)
@@ -1044,6 +1060,7 @@ static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid)list_move (&page->lru, &h->hugepage_activelist);set_page_refcounted (page); +ClearPageHugeFreed (page); h->free_huge_pages--; h->free_huge_pages_node[nid]--;return page;
@@ -1505,6 +1522,7 @@ static void prep_new_huge_page(struct hstate *h, struct page *page, int nid)spin_lock (&hugetlb_lock); h->nr_huge_pages++; h->nr_huge_pages_node[nid]++; +ClearPageHugeFreed (page);spin_unlock (&hugetlb_lock); }
@@ -1755,6 +1773,7 @@ int dissolve_free_huge_page(struct page *page) {int rc = -EBUSY ; +retry:/* Not to disrupt normal path by vainly holding hugetlb_lock */ if (!PageHuge (page))return 0 ;
@@ -1771,6 +1790,26 @@ int dissolve_free_huge_page(struct page *page)int nid =page_to_nid (head);if (h->free_huge_pages - h->resv_huge_pages ==0 ) goto out; + +/* +* We should make sure that the page is already on the free list +* when it is dissolved. +*/ +if (unlikely (!PageHugeFreed (head))) { +spin_unlock (&hugetlb_lock); +cond_resched (); + +/* +* Theoretically, we should return -EBUSY when we +* encounter this race. In fact, we have a chance +* to successfully dissolve the page if we do a +* retry. Because the race window is quite small. +* If we seize this opportunity, it is an optimization +* for increasing the success rate of dissolving page. +*/ + goto retry; + } +/* * Move PageHWPoison flag from head page to the raw error page, * which makes any subpages rather than the error page reusable.
Hi Greg,
Quoted 5 lines · On Mon, Feb 08, 2021 at 12:01:08PM +0100, [email protected] wrote:
The patch below does not apply to the 4.4-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.
Here is the backport.
--
Regards
Sudip
From 9585297e3f6ce7aafd72f7625c0eb6f2279558d2 Mon Sep 17 00:00:00 2001
From: Muchun Song <[email protected]>
Date: Thu, 4 Feb 2021 18:32:06 -0800
Subject: [PATCH] mm: hugetlb: fix a race between freeing and dissolving the
page
commit 7ffddd499ba6122b1a07828f023d1d67629aa017 upstream
There is a race condition between __free_huge_page() and dissolve_free_huge_page().
CPU0: CPU1:
// page_count(page) == 1
put_page(page)
__free_huge_page(page)
dissolve_free_huge_page(page)
spin_lock(&hugetlb_lock)
// PageHuge(page) && !page_count(page)
update_and_free_page(page)
// page is freed to the buddy
spin_unlock(&hugetlb_lock)
spin_lock(&hugetlb_lock)
clear_page_huge_active(page)
enqueue_huge_page(page)
// It is wrong, the page is already freed
spin_unlock(&hugetlb_lock)The race window is between put_page() and dissolve_free_huge_page().
We should make sure that the page is already on the free list when it is dissolved.
As a result __free_huge_page would corrupt page(s) already in the buddy allocator.
--- mm/hugetlb.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+)
mm/hugetlb.c +39 -0
@@ -66,6 +66,21 @@ DEFINE_SPINLOCK(hugetlb_lock);static int num_fault_mutexes;struct mutex *hugetlb_fault_mutex_table____cacheline_aligned_in_smp ; +static inline bool PageHugeFreed (struct page *head) +{ +return page_private (head +4 ) ==-1UL ; +} + +static inline void SetPageHugeFreed (struct page *head) +{ +set_page_private (head +4 ,-1UL ); +} + +static inline void ClearPageHugeFreed (struct page *head) +{ +set_page_private (head +4 ,0 ); +} +/* Forward declaration */ static int hugetlb_acct_memory (struct hstate *h,long delta);
@@ -841,6 +856,7 @@ static void enqueue_huge_page(struct hstate *h, struct page *page)list_move (&page->lru, &h->hugepage_freelists[nid]); h->free_huge_pages++; h->free_huge_pages_node[nid]++; +SetPageHugeFreed (page); }static struct page *dequeue_huge_page_node (struct hstate *h,int nid)
@@ -858,6 +874,7 @@ static struct page *dequeue_huge_page_node(struct hstate *h, int nid)return NULL ;list_move (&page->lru, &h->hugepage_activelist);set_page_refcounted (page); +ClearPageHugeFreed (page); h->free_huge_pages--; h->free_huge_pages_node[nid]--;return page;
@@ -1266,6 +1283,7 @@ static void prep_new_huge_page(struct hstate *h, struct page *page, int nid)set_hugetlb_cgroup (page,NULL ); h->nr_huge_pages++; h->nr_huge_pages_node[nid]++; +ClearPageHugeFreed (page);spin_unlock (&hugetlb_lock);put_page (page);/* free it into the hugepage allocator */ }
@@ -1424,11 +1442,32 @@ static int free_pool_huge_page(struct hstate *h, nodemask_t *nodes_allowed,*/ static void dissolve_free_huge_page (struct page *page) { +retry:spin_lock (&hugetlb_lock);if (PageHuge (page) && !page_count (page)) {struct page *head =compound_head (page);struct hstate *h =page_hstate (head);int nid =page_to_nid (head); + +/* +* We should make sure that the page is already on the free list +* when it is dissolved. +*/ +if (unlikely (!PageHugeFreed (head))) { +spin_unlock (&hugetlb_lock); +cond_resched (); + +/* +* Theoretically, we should return -EBUSY when we +* encounter this race. In fact, we have a chance +* to successfully dissolve the page if we do a +* retry. Because the race window is quite small. +* If we seize this opportunity, it is an optimization +* for increasing the success rate of dissolving page. +*/ + goto retry; + } +list_del (&head->lru); h->free_huge_pages--; h->free_huge_pages_node[nid]--;
Signature · 1 line
2.30.0
Greg KH
maintainerQuoted 10 lines · Sudip Mukherjee
Hi Greg,
Quoted 5 lines · On Mon, Feb 08, 2021 at 12:01:08PM +0100, [email protected] wrote:
The patch below does not apply to the 4.4-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.Here is the backport.
All now queued up, thanks.
greg k-h