Skip to contentSkip to conversation

mm: swap: properly update readahead statistics in unuse_pte_range()

PATCH Superseded Andrea Righi 3 participants 5 messages Apr 16, 2020 – Apr 18, 2020

Subsystems (2)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 mm: swap: properly update readahead statistics in unuse_pte_range()

In unuse_pte_range() we blindly swap-in pages without checking if the swap entry is already present in the swap cache.

By doing this, the hit/miss ratio used by the swap readahead heuristic is not properly updated and this leads to non-optimal performance during swapoff.

Tracing the distribution of the readahead size returned by the swap readahead heuristic during swapoff shows that a small readahead size is used most of the time as if we had only misses (this happens both with cluster and vma readahead), for example:

r::swapin_nr_pages(unsigned long offset):unsigned long:$retval

        COUNT      EVENT
        36948      $retval = 8
        44151      $retval = 4
        49290      $retval = 1
        527771     $retval = 2

Checking if the swap entry is present in the swap cache, instead, allows to properly update the readahead statistics and the heuristic behaves in a better way during swapoff, selecting a bigger readahead size:

r::swapin_nr_pages(unsigned long offset):unsigned long:$retval

        COUNT      EVENT
        1618       $retval = 1
        4960       $retval = 2
        41315      $retval = 4
        103521     $retval = 8

In terms of swapoff performance the result is the following:

Testing environment
===================

- Host:

   CPU: 1.8GHz Intel Core i7-8565U (quad-core, 8MB cache)
   HDD: PC401 NVMe SK hynix 512GB
   MEM: 16GB

- Guest (kvm):
8GB of RAM
virtio block driver
16GB swap file on ext4 (/swapfile)

Test case
=========
- allocate 85% of memory
- systemctl hibernate to force all the pages to be swapped-out to the swap file
- resume the system
- measure the time that swapoff takes to complete:

   # /usr/bin/time swapoff /swapfile

Result (swapoff time)
======
5.6 vanilla 5.6 w/ this patch
----------- -----------------

cluster-readahead      22.09s              12.19s
    vma-readahead      18.20s              15.33s

---
Changes in v3:
- properly update swap readahead statistics instead of forcing a fixed-size readahead

 mm/swapfile.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
mm/swapfile.c +8 -4
index 5871a2aa86a5..f8bf926c9c8f 100644
@@ -1937,10 +1937,14 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
 
 		pte_unmap(pte);
 		swap_map = &si->swap_map[offset];
-		vmf.vma = vma;
-		vmf.address = addr;
-		vmf.pmd = pmd;
-		page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, &vmf);
+		page = lookup_swap_cache(entry, vma, addr);
+		if (!page) {
+			vmf.vma = vma;
+			vmf.address = addr;
+			vmf.pmd = pmd;
+			page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE,
+						&vmf);
+		}
 		if (!page) {
 			if (*swap_map == 0 || *swap_map == SWAP_MAP_BAD)
 				goto try_next;
Signature · 1 line
2.25.1
#1 lore raw
Quoted 6 lines · Andrea Righi

In unuse_pte_range() we blindly swap-in pages without checking if the swap entry is already present in the swap cache.

By doing this, the hit/miss ratio used by the swap readahead heuristic is not properly updated and this leads to non-optimal performance during swapoff.

It's more important to describe why we need this patch in the patch description. So, please add some information about your use case. And please focus on the technical part instead of the business part.

Quoted 47 lines

Tracing the distribution of the readahead size returned by the swap readahead heuristic during swapoff shows that a small readahead size is used most of the time as if we had only misses (this happens both with cluster and vma readahead), for example:

r::swapin_nr_pages(unsigned long offset):unsigned long:$retval

        COUNT      EVENT
        36948      $retval = 8
        44151      $retval = 4
        49290      $retval = 1
        527771     $retval = 2

Checking if the swap entry is present in the swap cache, instead, allows to properly update the readahead statistics and the heuristic behaves in a better way during swapoff, selecting a bigger readahead size:

r::swapin_nr_pages(unsigned long offset):unsigned long:$retval

        COUNT      EVENT
        1618       $retval = 1
        4960       $retval = 2
        41315      $retval = 4
        103521     $retval = 8

In terms of swapoff performance the result is the following:

Testing environment
===================

- Host:

   CPU: 1.8GHz Intel Core i7-8565U (quad-core, 8MB cache)
   HDD: PC401 NVMe SK hynix 512GB
   MEM: 16GB

- Guest (kvm):
8GB of RAM
virtio block driver
16GB swap file on ext4 (/swapfile)

Test case
=========
- allocate 85% of memory
- systemctl hibernate to force all the pages to be swapped-out to the swap file
- resume the system
- measure the time that swapoff takes to complete:

   # /usr/bin/time swapoff /swapfile
Quoted 8 lines

Result (swapoff time)
======
5.6 vanilla 5.6 w/ this patch
----------- -----------------

cluster-readahead      22.09s              12.19s
    vma-readahead      18.20s              15.33s

Thanks! But you don't need to do this. You can add my Reviewed-by after we have finished the work on patch description.

Best Regards,
Huang, Ying

maintainer
#2 lore raw
Quoted 5 lines · Ying
Quoted 3 lines · Andrea Righi

In unuse_pte_range() we blindly swap-in pages without checking if the swap entry is already present in the swap cache.

Quoted 7 lines
Quoted 3 lines

By doing this, the hit/miss ratio used by the swap readahead heuristic is not properly updated and this leads to non-optimal performance during swapoff.

It's more important to describe why we need this patch in the patch description. So, please add some information about your use case. And please focus on the technical part instead of the business part.

Confused. I thought the changelog was quite good. If "business part" means "end user effect of the patch" then that's a very important thing.

Quoted 2 lines

Thanks! But you don't need to do this. You can add my Reviewed-by after we have finished the work on patch description.

Can you be more specific about how you want this changed?

Reviewed-by
#3 lore raw
Quoted 11 lines · Andrew Morton
Quoted 9 lines · Ying
Quoted 6 lines · Andrea Righi

In unuse_pte_range() we blindly swap-in pages without checking if the swap entry is already present in the swap cache.

By doing this, the hit/miss ratio used by the swap readahead heuristic is not properly updated and this leads to non-optimal performance during swapoff.

Quoted 7 lines
Quoted 3 lines

It's more important to describe why we need this patch in the patch description. So, please add some information about your use case. And please focus on the technical part instead of the business part.

Confused. I thought the changelog was quite good. If "business part" means "end user effect of the patch" then that's a very important thing.

Previously, Andrea has described their use case in the cloud environment to hiberate the guest and swapoff after resuming. So swapoff performance is important for them. I think that should be included. For the business part, I mean something like "Ubuntu used in AWS EC2", I think that isn't important for the patch description.

Quoted 4 lines
Quoted 2 lines

Thanks! But you don't need to do this. You can add my Reviewed-by after we have finished the work on patch description.

Can you be more specific about how you want this changed?

Please use

Best Regards,
Huang, Ying

#4 lore raw
Quoted 21 lines · Ying
Quoted 11 lines · Andrew Morton
Quoted 9 lines · Ying
Quoted 6 lines · Andrea Righi

In unuse_pte_range() we blindly swap-in pages without checking if the swap entry is already present in the swap cache.

By doing this, the hit/miss ratio used by the swap readahead heuristic is not properly updated and this leads to non-optimal performance during swapoff.

Quoted 7 lines
Quoted 3 lines

It's more important to describe why we need this patch in the patch description. So, please add some information about your use case. And please focus on the technical part instead of the business part.

Confused. I thought the changelog was quite good. If "business part" means "end user effect of the patch" then that's a very important thing.

Quoted 5 lines

Previously, Andrea has described their use case in the cloud environment to hiberate the guest and swapoff after resuming. So swapoff performance is important for them. I think that should be included. For the business part, I mean something like "Ubuntu used in AWS EC2", I think that isn't important for the patch description.

I just sent a v4 of this patch adding "conclusion" section in the description to better explain the purpose of this patch. Let me know if you have any comment on that.

Thanks,
-Andrea