Skip to contentSkip to conversation

Patch "mm, swap: fix race between swap count continuation operations" has been added to the 4.13-stable tree

PATCH [email protected] 1 participant 1 message Nov 5, 2017

Subsystems (3)
lore
Share & tools
lore

[email protected]

author maintainer Applied
#0 lore raw

This is a note to let you know that I've just added the patch titled

mm, swap: fix race between swap count continuation operations

to the 4.13-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
mm-swap-fix-race-between-swap-count-continuation-operations.patch
and it can be found in the queue-4.13 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From 2628bd6fc052bd85e9864dae4de494d8a6313391 Mon Sep 17 00:00:00 2001

From: Huang Ying <[email protected]>
Date: Thu, 2 Nov 2017 15:59:50 -0700
Subject: mm, swap: fix race between swap count continuation operations

From: Huang Ying <[email protected]>

commit 2628bd6fc052bd85e9864dae4de494d8a6313391 upstream.

One page may store a set of entries of the sis->swap_map (swap_info_struct->swap_map) in multiple swap clusters.

If some of the entries has sis->swap_map[offset] > SWAP_MAP_MAX, multiple pages will be used to store the set of entries of the sis->swap_map. And the pages are linked with page->lru. This is called swap count continuation. To access the pages which store the set of entries of the sis->swap_map simultaneously, previously, sis->lock is used. But to improve the scalability of __swap_duplicate(), swap cluster lock may be used in swap_count_continued() now. This may race with add_swap_count_continuation() which operates on a nearby swap cluster, in which the sis->swap_map entries are stored in the same page.

The race can cause wrong swap count in practice, thus cause unfreeable swap entries or software lockup, etc.

To fix the race, a new spin lock called cont_lock is added to struct swap_info_struct to protect the swap count continuation page list. This is a lock at the swap device level, so the scalability isn't very well. But it is still much better than the original sis->lock, because it is only acquired/released when swap count continuation is used. Which is considered rare in practice. If it turns out that the scalability becomes an issue for some workloads, we can split the lock into some more fine grained locks.

Fixes 235b62176712 ("mm/swap: add cluster lock")
Cc Johannes Weiner <[email protected]>
Cc Shaohua Li <[email protected]>
Cc Tim Chen <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Aaron Lu <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Andi Kleen <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Hugh Dickins <[email protected]>
---
 include/linux/swap.h |    4 ++++
 mm/swapfile.c        |   23 +++++++++++++++++------
 2 files changed, 21 insertions(+), 6 deletions(-)
include/linux/swap.h +4 -0
@@ -246,6 +246,10 @@ struct swap_info_struct {
 					 * both locks need hold, hold swap_lock
 					 * first.
 					 */
+	spinlock_t cont_lock;		/*
+					 * protect swap count continuation page
+					 * list.
+					 */
 	struct work_struct discard_work; /* discard worker */
 	struct swap_cluster_list discard_clusters; /* discard clusters list */
 };
mm/swapfile.c +17 -6
@@ -2635,6 +2635,7 @@ static struct swap_info_struct *alloc_sw
 	p->flags = SWP_USED;
 	spin_unlock(&swap_lock);
 	spin_lock_init(&p->lock);
+	spin_lock_init(&p->cont_lock);
 
 	return p;
 }
@@ -3307,6 +3308,7 @@ int add_swap_count_continuation(swp_entr
 	head = vmalloc_to_page(si->swap_map + offset);
 	offset &= ~PAGE_MASK;
 
+	spin_lock(&si->cont_lock);
 	/*
 	 * Page allocation does not initialize the page's lru field,
 	 * but it does always reset its private field.
@@ -3326,7 +3328,7 @@ int add_swap_count_continuation(swp_entr
 		 * a continuation page, free our allocation and use this one.
 		 */
 		if (!(count & COUNT_CONTINUED))
-			goto out;
+			goto out_unlock_cont;
 
 		map = kmap_atomic(list_page) + offset;
 		count = *map;
@@ -3337,11 +3339,13 @@ int add_swap_count_continuation(swp_entr
 		 * free our allocation and use this one.
 		 */
 		if ((count & ~COUNT_CONTINUED) != SWAP_CONT_MAX)
-			goto out;
+			goto out_unlock_cont;
 	}
 
 	list_add_tail(&page->lru, &head->lru);
 	page = NULL;			/* now it's attached, don't free it */
+out_unlock_cont:
+	spin_unlock(&si->cont_lock);
 out:
 	unlock_cluster(ci);
 	spin_unlock(&si->lock);
@@ -3366,6 +3370,7 @@ static bool swap_count_continued(struct
 	struct page *head;
 	struct page *page;
 	unsigned char *map;
+	bool ret;
 
 	head = vmalloc_to_page(si->swap_map + offset);
 	if (page_private(head) != SWP_CONTINUED) {
@@ -3373,6 +3378,7 @@ static bool swap_count_continued(struct
 		return false;		/* need to add count continuation */
 	}
 
+	spin_lock(&si->cont_lock);
 	offset &= ~PAGE_MASK;
 	page = list_entry(head->lru.next, struct page, lru);
 	map = kmap_atomic(page) + offset;
@@ -3393,8 +3399,10 @@ static bool swap_count_continued(struct
 		if (*map == SWAP_CONT_MAX) {
 			kunmap_atomic(map);
 			page = list_entry(page->lru.next, struct page, lru);
-			if (page == head)
-				return false;	/* add count continuation */
+			if (page == head) {
+				ret = false;	/* add count continuation */
+				goto out;
+			}
 			map = kmap_atomic(page) + offset;
 init_map:		*map = 0;		/* we didn't zero the page */
 		}
@@ -3407,7 +3415,7 @@ init_map:		*map = 0;		/* we didn't zero
 			kunmap_atomic(map);
 			page = list_entry(page->lru.prev, struct page, lru);
 		}
-		return true;			/* incremented */
+		ret = true;			/* incremented */
 
 	} else {				/* decrementing */
 		/*
@@ -3433,8 +3441,11 @@ init_map:		*map = 0;		/* we didn't zero
 			kunmap_atomic(map);
 			page = list_entry(page->lru.prev, struct page, lru);
 		}
-		return count == COUNT_CONTINUED;
+		ret = count == COUNT_CONTINUED;
 	}
+out:
+	spin_unlock(&si->cont_lock);
+	return ret;
 }
 
 /*

Patches currently in stable-queue which might be from [email protected] are

queue-4.13/mm-swap-fix-race-between-swap-count-continuation-operations.patch