Skip to contentSkip to conversation

swap: cleanup get/put_swap_device usage

PATCH In review Huang Ying 3 participants 5 messages May 16, 2023 – May 22, 2023

Subsystems (2)
lore
Share & tools
lore

Huang Ying

author
#0 lore raw

PATCH 1/1 swap: cleanup get/put_swap_device usage

The general rule to use a swap entry is as follows.

When we get a swap entry, if there isn't some other way to prevent swapoff, such as page lock for swap cache, page table lock, etc., the swap entry may become invalid because of swapoff. Then, we need to enclose all swap related functions with get_swap_device() and put_swap_device(), unless the swap functions call get/put_swap_device() by themselves.

Add the rule as comments of get_swap_device(), and cleanup some functions which call get/put_swap_device().

1. Enlarge the get/put_swap_device() protection range in __read_swap_cache_async(). This makes the function a little easier to be understood because we don't need to consider swapoff. And this makes it possible to remove get/put_swap_device() calling in some function called by __read_swap_cache_async().

2. Remove get/put_swap_device() in __swap_count(). Which is call in do_swap_page() only, which encloses the call with get/put_swap_device() already.

3. Remove get/put_swap_device() in __swp_swapcount(). Which is call in __read_swap_cache_async() only, which encloses the call with get/put_swap_device() already.

4. Remove get/put_swap_device() in __swap_duplicate(). Which is called by

- swap_shmem_alloc(): the swap cache is locked.

- copy_nonpresent_pte() -> swap_duplicate() and try_to_unmap_one() -> swap_duplicate(): the page table lock is held.

- __read_swap_cache_async() -> swapcache_prepare(): enclosed with get/put_swap_device() already.

Other get/put_swap_device() usages are checked too.

Cc David Hildenbrand <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Tim Chen <[email protected]>
Cc Yang Shi <[email protected]>
Cc Yu Zhao <[email protected]>
---
 include/linux/swap.h |  4 ++--
 mm/swap_state.c      | 33 ++++++++++++++++++++-----------
 mm/swapfile.c        | 47 ++++++++++++--------------------------------
 3 files changed, 37 insertions(+), 47 deletions(-)
include/linux/swap.h +2 -2
index 3c69cb653cb9..f6bd51aa05ea 100644
@@ -512,7 +512,7 @@ int find_first_swap(dev_t *device);
 extern unsigned int count_swap_pages(int, int);
 extern sector_t swapdev_block(int, pgoff_t);
 extern int __swap_count(swp_entry_t entry);
-extern int __swp_swapcount(swp_entry_t entry);
+extern int swap_swapcount(struct swap_info_struct *si, swp_entry_t entry);
 extern int swp_swapcount(swp_entry_t entry);
 extern struct swap_info_struct *page_swap_info(struct page *);
 extern struct swap_info_struct *swp_swap_info(swp_entry_t entry);
@@ -590,7 +590,7 @@ static inline int __swap_count(swp_entry_t entry)
 	return 0;
 }
 
-static inline int __swp_swapcount(swp_entry_t entry)
+static inline int swap_swapcount(struct swap_info_struct *si, swp_entry_t entry)
 {
 	return 0;
 }
mm/swap_state.c +22 -11
index b76a65ac28b3..a1028fe7214e 100644
@@ -417,9 +417,13 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
 {
 	struct swap_info_struct *si;
 	struct folio *folio;
+	struct page *page;
 	void *shadow = NULL;
 
 	*new_page_allocated = false;
+	si = get_swap_device(entry);
+	if (!si)
+		return NULL;
 
 	for (;;) {
 		int err;
@@ -428,14 +432,12 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
 		 * called after swap_cache_get_folio() failed, re-calling
 		 * that would confuse statistics.
 		 */
-		si = get_swap_device(entry);
-		if (!si)
-			return NULL;
 		folio = filemap_get_folio(swap_address_space(entry),
 						swp_offset(entry));
-		put_swap_device(si);
-		if (!IS_ERR(folio))
-			return folio_file_page(folio, swp_offset(entry));
+		if (!IS_ERR(folio)) {
+			page = folio_file_page(folio, swp_offset(entry));
+			goto got_page;
+		}
 
 		/*
 		 * Just skip read ahead for unused swap slot.
@@ -445,8 +447,8 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
 		 * as SWAP_HAS_CACHE.  That's done in later part of code or
 		 * else swap_off will be aborted if we return NULL.
 		 */
-		if (!__swp_swapcount(entry) && swap_slot_cache_enabled)
-			return NULL;
+		if (!swap_swapcount(si, entry) && swap_slot_cache_enabled)
+			goto fail;
 
 		/*
 		 * Get a new page to read into from swap.  Allocate it now,
@@ -455,7 +457,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
 		 */
 		folio = vma_alloc_folio(gfp_mask, 0, vma, addr, false);
 		if (!folio)
-			return NULL;
+                        goto fail;
 
 		/*
 		 * Swap entry may have been freed since our caller observed it.
@@ -466,7 +468,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
 
 		folio_put(folio);
 		if (err != -EEXIST)
-			return NULL;
+			goto fail;
 
 		/*
 		 * We might race against __delete_from_swap_cache(), and
@@ -500,12 +502,17 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
 	/* Caller will initiate read into locked folio */
 	folio_add_lru(folio);
 	*new_page_allocated = true;
-	return &folio->page;
+	page = &folio->page;
+got_page:
+	put_swap_device(si);
+	return page;
 
 fail_unlock:
 	put_swap_folio(folio, entry);
 	folio_unlock(folio);
 	folio_put(folio);
+fail:
+	put_swap_device(si);
 	return NULL;
 }
 
@@ -514,6 +521,10 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
  * and reading the disk if it is not already cached.
  * A failure return means that either the page allocation failed or that
  * the swap entry is no longer in use.
+ *
+ * get/put_swap_device() aren't needed to call this function, because
+ * __read_swap_cache_async() call them and swap_readpage() holds the
+ * swap cache folio lock.
  */
 struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
 				   struct vm_area_struct *vma,
mm/swapfile.c +13 -34

91 lines not on this page · raw message

Signature · 1 line
2.39.2
#1 lore raw
Quoted 31 lines · Huang Ying

The general rule to use a swap entry is as follows.

When we get a swap entry, if there isn't some other way to prevent swapoff, such as page lock for swap cache, page table lock, etc., the swap entry may become invalid because of swapoff. Then, we need to enclose all swap related functions with get_swap_device() and put_swap_device(), unless the swap functions call get/put_swap_device() by themselves.

Add the rule as comments of get_swap_device(), and cleanup some functions which call get/put_swap_device().

1. Enlarge the get/put_swap_device() protection range in __read_swap_cache_async(). This makes the function a little easier to be understood because we don't need to consider swapoff. And this makes it possible to remove get/put_swap_device() calling in some function called by __read_swap_cache_async().

2. Remove get/put_swap_device() in __swap_count(). Which is call in do_swap_page() only, which encloses the call with get/put_swap_device() already.

3. Remove get/put_swap_device() in __swp_swapcount(). Which is call in __read_swap_cache_async() only, which encloses the call with get/put_swap_device() already.

4. Remove get/put_swap_device() in __swap_duplicate(). Which is called by

- swap_shmem_alloc(): the swap cache is locked.

Quoted 7 lines

- copy_nonpresent_pte() -> swap_duplicate() and try_to_unmap_one() -> swap_duplicate(): the page table lock is held.

- __read_swap_cache_async() -> swapcache_prepare(): enclosed with get/put_swap_device() already.

Other get/put_swap_device() usages are checked too.

I suggest splitting this patch up into logical pieces as outlined here by you already.

Signature · 3 lines
Thanks,

David / dhildenb
#2 lore raw
Quoted 32 lines · David Hildenbrand
Quoted 22 lines · Huang Ying

The general rule to use a swap entry is as follows.
When we get a swap entry, if there isn't some other way to prevent swapoff, such as page lock for swap cache, page table lock, etc., the swap entry may become invalid because of swapoff. Then, we need to enclose all swap related functions with get_swap_device() and put_swap_device(), unless the swap functions call get/put_swap_device() by themselves.
Add the rule as comments of get_swap_device(), and cleanup some functions which call get/put_swap_device().
1. Enlarge the get/put_swap_device() protection range in __read_swap_cache_async(). This makes the function a little easier to be understood because we don't need to consider swapoff. And this makes it possible to remove get/put_swap_device() calling in some function called by __read_swap_cache_async().
2. Remove get/put_swap_device() in __swap_count(). Which is call in do_swap_page() only, which encloses the call with get/put_swap_device() already.
3. Remove get/put_swap_device() in __swp_swapcount(). Which is call in __read_swap_cache_async() only, which encloses the call with get/put_swap_device() already.
4. Remove get/put_swap_device() in __swap_duplicate(). Which is called

Quoted 8 lines

by
- swap_shmem_alloc(): the swap cache is locked.
- copy_nonpresent_pte() -> swap_duplicate() and try_to_unmap_one() ->
swap_duplicate(): the page table lock is held.
- __read_swap_cache_async() -> swapcache_prepare(): enclosed with get/put_swap_device() already.
Other get/put_swap_device() usages are checked too.

Quoted 2 lines

I suggest splitting this patch up into logical pieces as outlined here by you already.

OK. Will do that in the next version.

Best Regards,
Huang, Ying

maintainer
#3 lore raw
Quoted 34 lines · Ying
Quoted 32 lines · David Hildenbrand
Quoted 30 lines · Huang Ying

The general rule to use a swap entry is as follows.
When we get a swap entry, if there isn't some other way to prevent swapoff, such as page lock for swap cache, page table lock, etc., the swap entry may become invalid because of swapoff. Then, we need to enclose all swap related functions with get_swap_device() and put_swap_device(), unless the swap functions call get/put_swap_device() by themselves.
Add the rule as comments of get_swap_device(), and cleanup some functions which call get/put_swap_device().
1. Enlarge the get/put_swap_device() protection range in __read_swap_cache_async(). This makes the function a little easier to be understood because we don't need to consider swapoff. And this makes it possible to remove get/put_swap_device() calling in some function called by __read_swap_cache_async().
2. Remove get/put_swap_device() in __swap_count(). Which is call in do_swap_page() only, which encloses the call with get/put_swap_device() already.
3. Remove get/put_swap_device() in __swp_swapcount(). Which is call in __read_swap_cache_async() only, which encloses the call with get/put_swap_device() already.
4. Remove get/put_swap_device() in __swap_duplicate(). Which is called
by
- swap_shmem_alloc(): the swap cache is locked.
- copy_nonpresent_pte() -> swap_duplicate() and try_to_unmap_one() ->
swap_duplicate(): the page table lock is held.
- __read_swap_cache_async() -> swapcache_prepare(): enclosed with get/put_swap_device() already.
Other get/put_swap_device() usages are checked too.

Quoted 2 lines
Quoted 2 lines

I suggest splitting this patch up into logical pieces as outlined here by you already.

Agree with David here.

Quoted 2 lines

OK. Will do that in the next version.

Your patch make sense to me.

Looking forward to your next version.

BTW, no relat to your patch, but just when I look at your patch I notice is that we have too many swap count functions.
The naming scheme is very confusing.

1) swap_count(), just mask out SWAP_HAS_CACHE

2) __swap_count() the name with underscore suggest it is more internal. But __swap_count() calls swap_count(). It is basically swap_count() with device lookup.

3) swap_swapcount()
similar to __swap_count() but with cluster level locking if possible. otherwise fall back to device level locking.

4) __swp_swapcount()
swap_swapcount () with device lookup. not consider continuing. Again this function is more external while swap_swapcount() is more internal.

5) swp_swapcount() similar to __swp_swapcount()
exact count consider continue

We should have a more consistent naming regarding swap count. Device level, then cluster level, then entry level.

Also I consider the continuing is internal to the current swap index implementation. If we have alternative swap file implementation, we might not have count continuing at all.

Chris

#4 lore raw
Quoted 67 lines · Chris Li
Quoted 34 lines · Ying
Quoted 32 lines · David Hildenbrand
Quoted 30 lines · Huang Ying

The general rule to use a swap entry is as follows.
When we get a swap entry, if there isn't some other way to prevent swapoff, such as page lock for swap cache, page table lock, etc., the swap entry may become invalid because of swapoff. Then, we need to enclose all swap related functions with get_swap_device() and put_swap_device(), unless the swap functions call get/put_swap_device() by themselves.
Add the rule as comments of get_swap_device(), and cleanup some functions which call get/put_swap_device().
1. Enlarge the get/put_swap_device() protection range in __read_swap_cache_async(). This makes the function a little easier to be understood because we don't need to consider swapoff. And this makes it possible to remove get/put_swap_device() calling in some function called by __read_swap_cache_async().
2. Remove get/put_swap_device() in __swap_count(). Which is call in do_swap_page() only, which encloses the call with get/put_swap_device() already.
3. Remove get/put_swap_device() in __swp_swapcount(). Which is call in __read_swap_cache_async() only, which encloses the call with get/put_swap_device() already.
4. Remove get/put_swap_device() in __swap_duplicate(). Which is called
by
- swap_shmem_alloc(): the swap cache is locked.
- copy_nonpresent_pte() -> swap_duplicate() and try_to_unmap_one() ->
swap_duplicate(): the page table lock is held.
- __read_swap_cache_async() -> swapcache_prepare(): enclosed with get/put_swap_device() already.
Other get/put_swap_device() usages are checked too.

Quoted 2 lines
Quoted 2 lines

I suggest splitting this patch up into logical pieces as outlined here by you already.

Agree with David here.

Quoted 2 lines

OK. Will do that in the next version.

Your patch make sense to me.

Looking forward to your next version.

BTW, no relat to your patch, but just when I look at your patch I notice is that we have too many swap count functions.
The naming scheme is very confusing.

1) swap_count(), just mask out SWAP_HAS_CACHE

2) __swap_count() the name with underscore suggest it is more internal. But __swap_count() calls swap_count(). It is basically swap_count() with device lookup.

3) swap_swapcount()
similar to __swap_count() but with cluster level locking if possible. otherwise fall back to device level locking.

4) __swp_swapcount()
swap_swapcount () with device lookup. not consider continuing. Again this function is more external while swap_swapcount() is more internal.

Quoted 5 lines

5) swp_swapcount() similar to __swp_swapcount()
exact count consider continue

We should have a more consistent naming regarding swap count. Device level, then cluster level, then entry level.

Yes. The original naming is confusing.

Quoted 3 lines

Also I consider the continuing is internal to the current swap index implementation. If we have alternative swap file implementation, we might not have count continuing at all.

There's some difficulties to hide continuation completely. For example, we want to call add_swap_count_continuation() in non-atomic context in copy_pte_range(), while the fast path calls swap_duplicate() in atomic context (via copy_nonpresent_pte()).

Best Regards,
Huang, Ying