mm,swap: skip swap readahead for instant IO (like zswap)

mm/swap_state.c static unsigned long swapin_nr_pages(unsigned long offset) · line 605

Quoted 9 lines · Rik van Riel

mm/swap_state.c +9 -0

+static struct page *swap_cluster_read_one(swp_entry_t entry,
+		unsigned long offset, gfp_t gfp_mask,
+		struct vm_area_struct *vma, unsigned long addr, bool readahead)
+{
+	bool page_allocated;
+	struct page *page;
+
+	page =__read_swap_cache_async(swp_entry(swp_type(entry), offset),
+				      gfp_mask, vma, addr, &page_allocated);

Missing whitespace after the "=".

mm/swap_state.c static unsigned long swapin_nr_pages(unsigned long offset) · line 617

Quoted 12 lines

mm/swap_state.c +12 -0

+	if (!page)
+		return NULL;
+	if (page_allocated) {
+		swap_readpage(page, false);
+		if (readahead) {
+			SetPageReadahead(page);
+			count_vm_event(SWAP_RA);
+		}
+	}
+	put_page(page);
+	return page;
+}

I think swap_vma_readahead can be switched to your new helper pretty trivially as well, as could many of the users of read_swap_cache_async.

Check whether a swap page was obtained instantaneously, for example because it is in zswap, or on a very fast IO device which uses busy waiting, and we did not wait on IO to swap in this page.
If no IO was needed to get the swap page we want, kicking off readahead on surrounding swap pages is likely to be counterproductive, because the extra loads will cause additional latency, use up extra memory, and chances are the surrounding pages in swap are just as fast to load as this one, making readahead pointless.

Signed-off-by Rik van Riel <[email protected]>

---
 mm/swap_state.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

mm/swap_state.c +0 -0

index aacb9ba53f63..6919f9d5fe88 100644

mm/swap_state.c +7 -1

 {
+       struct page *page;
        unsigned long entry_offset = swp_offset(entry);
        unsigned long offset = entry_offset;
        unsigned long start_offset, end_offset;

@@ -668,11 +669,18 @@ struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask,
                end_offset = si->max - 1;
 
        blk_start_plug(&plug);
+       /* If we read the page without waiting on IO, skip readahead. */
+       page = swap_cluster_read_one(entry, offset, gfp_mask, vma, addr, false);
+       if (page && PageUptodate(page))
+               goto skip_unplug;
+
+       /* Ok, do the async read-ahead now. */
        for (offset = start_offset; offset <= end_offset ; offset++) {
-               /* Ok, do the async read-ahead now */

Quoted 18 lines · Rik van Riel

Check whether a swap page was obtained instantaneously, for example because it is in zswap, or on a very fast IO device which uses busy waiting, and we did not wait on IO to swap in this page. If no IO was needed to get the swap page we want, kicking off readahead
on surrounding swap pages is likely to be counterproductive, because the
extra loads will cause additional latency, use up extra memory, and chances
are the surrounding pages in swap are just as fast to load as this one,
making readahead pointless.

Signed-off-by Rik van Riel <[email protected]>

---
 mm/swap_state.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

mm/swap_state.c static struct page · line 640

Quoted 9 lines

mm/swap_state.c +0 -0

index aacb9ba53f63..6919f9d5fe88 100644
@@ -637,6 +637,7 @@ static struct page
 *swap_cluster_read_one(swp_entry_t entry,
 struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t
 gfp_mask,
                                struct vm_fault *vmf)

Why not do this for swap_vma_readahead()
too? swap_cluster_read_one()
can be used in swap_vma_readahead() too.

Both with frontswap/zswap, and with some extremely fast IO devices, swap IO will be done before the "asynchronous" swap_readpage() call has returned.

In that case, doing swap readahead only wastes memory, increases latency, and increases the chances of needing to evict something more useful from memory. In that case, just skip swap readahead.

Quoted 4 lines · On Mon, 21 Sep 2020 22:01:46 -0400 Rik van Riel <[email protected]> wrote:

Both with frontswap/zswap, and with some extremely fast IO devices, swap IO will be done before the "asynchronous" swap_readpage() call has returned.

Quoted 4 lines

In that case, doing swap readahead only wastes memory, increases latency, and increases the chances of needing to evict something more
useful from memory. In that case, just skip swap readahead.

Any quantitative testing results?

Quoted 4 lines · Andrew Morton

On Mon, 21 Sep 2020 22:01:46 -0400 Rik van Riel <[email protected]> wrote:

Any quantitative testing results?

I have test results with a real workload now.

Without this patch, enabling zswap results in about an 8% increase in p99 request latency. With these patches, the latency penalty for enabling zswap is under 1%.