Skip to contentSkip to conversation

mm/zswap: store <PAGE_SIZE compression failed page as-is

PATCH Superseded SeongJae Park 6 participants 24 messages Aug 12, 2025 – Aug 18, 2025

Subsystems (2)

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm/zswap: store <PAGE_SIZE compression failed page as-is

When zswap writeback is enabled and it fails compressing a given page, the page is swapped out to the backing swap device. This behavior breaks the zswap's writeback LRU order, and hence users can experience unexpected latency spikes. If the page is compressed without failure, but results in a size of PAGE_SIZE, the LRU order is kept, but the decompression overhead for loading the page back on the later access is unnecessary.

Keep the LRU order and optimize unnecessary decompression overheads in those cases, by storing the original content as-is in zpool. The length field of zswap_entry will be set appropriately, as PAGE_SIZE, Hence whether it is saved as-is or not (whether decompression is unnecessary) is identified by 'zswap_entry->length == PAGE_SIZE'.

Because the uncompressed data is saved in zpool, same to the compressed ones, this introduces no change in terms of memory management including movability and migratability of involved pages.

This change is also not increasing per zswap entry metadata overhead. But as the number of incompressible pages increases, total zswap metadata overhead is proportionally increased. The overhead should not be problematic in usual cases, since the zswap metadata for single zswap entry is much smaller than PAGE_SIZE, and in common zswap use cases there should be a sufficient amount of compressible pages. Also it can be mitigated by the zswap writeback.

When the writeback is disabled, the additional overhead could be problematic. For the case, keep the current behavior that just returns the failure and let swap_writeout() put the page back to the active LRU list in the case.

Knowing how many compression failures happened will be useful for future investigations. investigations. Add a new debugfs file, compress_fail, for the purpose.

Tests
-----

I tested this patch using a simple self-written microbenchmark that is available at GitHub[1]. You can reproduce the test I did by executing run_tests.sh of the repo on your system. Note that the repo's documentation is not good as of this writing, so you may need to read and use the code.

The basic test scenario is simple. Run a test program making artificial accesses to memory having artificial content under memory.high-set memory limit and measure how many accesses were made in given time.

The test program repeatedly and randomly access three anonymous memory regions. The regions are all 500 MiB size, and accessed in the same probability. Two of those are filled up with a simple content that can easily be compressed, while the remaining one is filled up with a content that read from /dev/urandom, which is easy to fail at compressing to <PAGE_SIZE size. The program runs for two minutes and prints out the number of accesses made every five seconds.

The test script runs the program under below seven configurations.

- 0: memory.high is set to 2 GiB, zswap is disabled.
- 1-1: memory.high is set to 1350 MiB, zswap is disabled.
- 1-2: On 1-1, zswap is enabled without this patch.
- 1-3: On 1-2, this patch is applied.

For all zswap enabled cases, zswap shrinker is enabled.

Configuration '0' is for showing the original memory performance. Configurations 1-1, 1-2 and 1-3 are for showing the performance of swap, zswap, and this patch under a level of memory pressure (~10% of working set).

Because the per-5 seconds performance is not very reliable, I measured the average of that for the last one minute period of the test program run. I also measured a few vmstat counters including zswpin, zswpout, zswpwb, pswpin and pswpout during the test runs.

The measurement results are as below. To save space, I show performance numbers that are normalized to that of the configuration '0' (no memory pressure), only. The averaged accesses per 5 seconds of configuration '0' was 36493417.75.

    config            0       1-1     1-2      1-3
    perf_normalized   1.0000  0.0057  0.0235   0.0367
    perf_stdev_ratio  0.0582  0.0652  0.0167   0.0346
    zswpin            0       0       3548424  1999335
    zswpout           0       0       3588817  2361689
    zswpwb            0       0       10214    340270
    pswpin            0       485806  772038   340967
    pswpout           0       649543  144773   340270

'perf_normalized' is the performance metric, normalized to that of configuration '0' (no pressure). 'perf_stdev_ratio' is the standard deviation of the averaged data points, as a ratio to the averaged metric

value.  For example, configuration '0' performance was showing 5.8%
stdev.  Configurations 1-1 and 1-3 were having about 6.5% and 6.1%
stdev.  Also the results were highly variable between multiple runs.  So

this result is not very stable but just showing ball park figures. Please keep this in your mind when reading these results.

Under about 10% of working set memory pressure, the performance was dropped to about 0.57% of no-pressure one, when the normal swap is used (1-1). Note that ~10% working set pressure is already extreme, at least on this test setup. No one would desire system setups that can degrade performance to 0.57% of the best case.

By turning zswap on (1-2), the performance was improved about 4x, resulting in about 2.35% of no-pressure one. Because of the incompressible pages in the third memory region, a significant amount of (non-zswap) swap I/O operations were made, though.

By applying this patch (1-3), about 56% performance improvement was made, resulting in about 3.67% of no-pressure one. Reduced pswpin of 1-3 compared to 1-2 let us see where this improvement came from.

Related Works
-------------

This is not an entirely new attempt. Nhat Pham and Takero Funaki tried very similar approaches in October 2023[2] and April 2024[3], respectively. The two approaches didn't get merged mainly due to the metadata overhead concern. I described why I think that shouldn't be a problem for this change, which is automatically disabled when writeback is disabled, at the beginning of this changelog.

This patch is not particularly different from those, and actually built upon those. I wrote this from scratch again, though. Hence adding Suggested-by tags for them. Actually Nhat first suggested this to me offlist.

Historically, writeback disabling was introduced partially as a way to solve the LRU order issue. Yosry pointed out[4] this is still suboptimal when the incompressible pages are cold, since the incompressible pages will continuously be tried to be zswapped out, and burn CPU cycles for compression attempts that will anyway fail. One imaginable solution for the problem is reusing the swapped-out page and its struct page to store in the zswap pool. But that's out of the scope of this patch.

[1] https://github.com/sjp38/eval_zswap/blob/master/run.sh
[2] https://lore.kernel.org/[email protected]
[3] https://lore.kernel.org/[email protected]
[4] https://lore.kernel.org/CAJD7tkZXS-UJVAFfvxJ0nNgTzWBiqepPYA4hEozi01_qktkitg@mail.gmail.com

---
Changes from v1
(https://lore.kernel.org/[email protected])
- Optimize out memcpy() per incompressible page saving, using k[un]map_local().
- Add a debugfs file for counting compression failures.
- Use a clear form of a ternary operation.
- Add the history of writeback disabling with a link.
- Wordsmith comments.

Changes from RFC v2
(https://lore.kernel.org/[email protected])
- Fix race conditions at decompressed pages identification.
- Remove the parameter and make saving as-is the default behavior.
- Open-code main changes.
- Clarify there is no memory management changes on the cover letter.
- Remove 20% pressure case from test results, since it is arguably too extreme and only adds confusion.
- Drop RFC tag.

Changes from RFC v1
(https://lore.kernel.org/[email protected])
- Consider PAGE_SIZE compression successes as failures.
- Use zpool for storing incompressible pages.
- Test with zswap shrinker enabled.
- Wordsmith changelog and comments.
- Add documentation of save_incompressible_pages parameter.

 mm/zswap.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)
mm/zswap.c +34 -2
index 3c0fd8a13718..0fb940d03268 100644
@@ -60,6 +60,8 @@ static u64 zswap_written_back_pages;
 static u64 zswap_reject_reclaim_fail;
 /* Store failed due to compression algorithm failure */
 static u64 zswap_reject_compress_fail;
+/* Compression into a size of <PAGE_SIZE failed */
+static u64 zswap_compress_fail;
 /* Compressed page was too big for the allocator to (optimally) store */
 static u64 zswap_reject_compress_poor;
 /* Load or writeback failed due to decompression failure */
@@ -976,8 +978,26 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
 	 */
 	comp_ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait);
 	dlen = acomp_ctx->req->dlen;
-	if (comp_ret)
-		goto unlock;
+
+	/*
+	 * If a page cannot be compressed into a size smaller than PAGE_SIZE,
+	 * save the content as is without a compression, to keep the LRU order
+	 * of writebacks.  If writeback is disabled, reject the page since it
+	 * only adds metadata overhead.  swap_writeout() will put the page back
+	 * to the active LRU list in the case.
+	 */
+	if (comp_ret || dlen >= PAGE_SIZE) {
+		zswap_compress_fail++;
+		if (mem_cgroup_zswap_writeback_enabled(
+					folio_memcg(page_folio(page)))) {
+			comp_ret = 0;
+			dlen = PAGE_SIZE;
+			dst = kmap_local_page(page);
+		} else {
+			comp_ret = comp_ret ? comp_ret : -EINVAL;
+			goto unlock;
+		}
+	}
 
 	zpool = pool->zpool;
 	gfp = GFP_NOWAIT | __GFP_NORETRY | __GFP_HIGHMEM | __GFP_MOVABLE;
@@ -990,6 +1010,8 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
 	entry->length = dlen;
 
 unlock:
+	if (dst != acomp_ctx->buffer)
+		kunmap_local(dst);
 	if (comp_ret == -ENOSPC || alloc_ret == -ENOSPC)
 		zswap_reject_compress_poor++;
 	else if (comp_ret)
@@ -1012,6 +1034,14 @@ static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio)
 	acomp_ctx = acomp_ctx_get_cpu_lock(entry->pool);
 	obj = zpool_obj_read_begin(zpool, entry->handle, acomp_ctx->buffer);
 
+	/* zswap entries of length PAGE_SIZE are not compressed. */
+	if (entry->length == PAGE_SIZE) {
+		memcpy_to_folio(folio, 0, obj, entry->length);
+		zpool_obj_read_end(zpool, entry->handle, obj);
+		acomp_ctx_put_unlock(acomp_ctx);
+		return true;
+	}
+
 	/*
 	 * zpool_obj_read_begin() might return a kmap address of highmem when
 	 * acomp_ctx->buffer is not used.  However, sg_init_one() does not
@@ -1809,6 +1839,8 @@ static int zswap_debugfs_init(void)
 			   zswap_debugfs_root, &zswap_reject_kmemcache_fail);
 	debugfs_create_u64("reject_compress_fail", 0444,
 			   zswap_debugfs_root, &zswap_reject_compress_fail);
+	debugfs_create_u64("compress_fail", 0444,
+			   zswap_debugfs_root, &zswap_compress_fail);
 	debugfs_create_u64("reject_compress_poor", 0444,
 			   zswap_debugfs_root, &zswap_reject_compress_poor);
 	debugfs_create_u64("decompress_fail", 0444,

base-commit: 44fa6646d975349f6499d1aeb0ed826680d0bb5c

Signature · 1 line
2.39.5
maintainer Acked-by on v2 1/1
#1 lore raw
Quoted 174 lines · SeongJae Park

When zswap writeback is enabled and it fails compressing a given page, the page is swapped out to the backing swap device. This behavior breaks the zswap's writeback LRU order, and hence users can experience unexpected latency spikes. If the page is compressed without failure, but results in a size of PAGE_SIZE, the LRU order is kept, but the decompression overhead for loading the page back on the later access is unnecessary.

Keep the LRU order and optimize unnecessary decompression overheads in those cases, by storing the original content as-is in zpool. The length field of zswap_entry will be set appropriately, as PAGE_SIZE, Hence whether it is saved as-is or not (whether decompression is unnecessary) is identified by 'zswap_entry->length == PAGE_SIZE'.

Because the uncompressed data is saved in zpool, same to the compressed ones, this introduces no change in terms of memory management including movability and migratability of involved pages.

This change is also not increasing per zswap entry metadata overhead. But as the number of incompressible pages increases, total zswap metadata overhead is proportionally increased. The overhead should not be problematic in usual cases, since the zswap metadata for single zswap entry is much smaller than PAGE_SIZE, and in common zswap use cases there should be a sufficient amount of compressible pages. Also it can be mitigated by the zswap writeback.

When the writeback is disabled, the additional overhead could be problematic. For the case, keep the current behavior that just returns the failure and let swap_writeout() put the page back to the active LRU list in the case.

Knowing how many compression failures happened will be useful for future investigations. investigations. Add a new debugfs file, compress_fail, for the purpose.

Tests
-----

I tested this patch using a simple self-written microbenchmark that is available at GitHub[1]. You can reproduce the test I did by executing run_tests.sh of the repo on your system. Note that the repo's documentation is not good as of this writing, so you may need to read and use the code.

The basic test scenario is simple. Run a test program making artificial accesses to memory having artificial content under memory.high-set memory limit and measure how many accesses were made in given time.

The test program repeatedly and randomly access three anonymous memory regions. The regions are all 500 MiB size, and accessed in the same probability. Two of those are filled up with a simple content that can easily be compressed, while the remaining one is filled up with a content that read from /dev/urandom, which is easy to fail at compressing to <PAGE_SIZE size. The program runs for two minutes and prints out the number of accesses made every five seconds.

The test script runs the program under below seven configurations.

- 0: memory.high is set to 2 GiB, zswap is disabled.
- 1-1: memory.high is set to 1350 MiB, zswap is disabled.
- 1-2: On 1-1, zswap is enabled without this patch.
- 1-3: On 1-2, this patch is applied.

For all zswap enabled cases, zswap shrinker is enabled.

Configuration '0' is for showing the original memory performance. Configurations 1-1, 1-2 and 1-3 are for showing the performance of swap, zswap, and this patch under a level of memory pressure (~10% of working set).

Because the per-5 seconds performance is not very reliable, I measured the average of that for the last one minute period of the test program run. I also measured a few vmstat counters including zswpin, zswpout, zswpwb, pswpin and pswpout during the test runs.

The measurement results are as below. To save space, I show performance numbers that are normalized to that of the configuration '0' (no memory pressure), only. The averaged accesses per 5 seconds of configuration '0' was 36493417.75.

    config            0       1-1     1-2      1-3
    perf_normalized   1.0000  0.0057  0.0235   0.0367
    perf_stdev_ratio  0.0582  0.0652  0.0167   0.0346
    zswpin            0       0       3548424  1999335
    zswpout           0       0       3588817  2361689
    zswpwb            0       0       10214    340270
    pswpin            0       485806  772038   340967
    pswpout           0       649543  144773   340270

'perf_normalized' is the performance metric, normalized to that of configuration '0' (no pressure). 'perf_stdev_ratio' is the standard deviation of the averaged data points, as a ratio to the averaged metric

value.  For example, configuration '0' performance was showing 5.8%
stdev.  Configurations 1-1 and 1-3 were having about 6.5% and 6.1%
stdev.  Also the results were highly variable between multiple runs.  So

this result is not very stable but just showing ball park figures. Please keep this in your mind when reading these results.

Under about 10% of working set memory pressure, the performance was dropped to about 0.57% of no-pressure one, when the normal swap is used (1-1). Note that ~10% working set pressure is already extreme, at least on this test setup. No one would desire system setups that can degrade performance to 0.57% of the best case.

By turning zswap on (1-2), the performance was improved about 4x, resulting in about 2.35% of no-pressure one. Because of the incompressible pages in the third memory region, a significant amount of (non-zswap) swap I/O operations were made, though.

By applying this patch (1-3), about 56% performance improvement was made, resulting in about 3.67% of no-pressure one. Reduced pswpin of 1-3 compared to 1-2 let us see where this improvement came from.

Related Works
-------------

This is not an entirely new attempt. Nhat Pham and Takero Funaki tried very similar approaches in October 2023[2] and April 2024[3], respectively. The two approaches didn't get merged mainly due to the metadata overhead concern. I described why I think that shouldn't be a problem for this change, which is automatically disabled when writeback is disabled, at the beginning of this changelog.

This patch is not particularly different from those, and actually built upon those. I wrote this from scratch again, though. Hence adding Suggested-by tags for them. Actually Nhat first suggested this to me offlist.

Historically, writeback disabling was introduced partially as a way to solve the LRU order issue. Yosry pointed out[4] this is still suboptimal when the incompressible pages are cold, since the incompressible pages will continuously be tried to be zswapped out, and burn CPU cycles for compression attempts that will anyway fail. One imaginable solution for the problem is reusing the swapped-out page and its struct page to store in the zswap pool. But that's out of the scope of this patch.

[1] https://github.com/sjp38/eval_zswap/blob/master/run.sh
[2] https://lore.kernel.org/[email protected]
[3] https://lore.kernel.org/[email protected]
[4] https://lore.kernel.org/CAJD7tkZXS-UJVAFfvxJ0nNgTzWBiqepPYA4hEozi01_qktkitg@mail.gmail.com

---
Changes from v1
(https://lore.kernel.org/[email protected])
- Optimize out memcpy() per incompressible page saving, using k[un]map_local().
- Add a debugfs file for counting compression failures.
- Use a clear form of a ternary operation.
- Add the history of writeback disabling with a link.
- Wordsmith comments.

Changes from RFC v2
(https://lore.kernel.org/[email protected])
- Fix race conditions at decompressed pages identification.
- Remove the parameter and make saving as-is the default behavior.
- Open-code main changes.
- Clarify there is no memory management changes on the cover letter.
- Remove 20% pressure case from test results, since it is arguably too extreme and only adds confusion.
- Drop RFC tag.

Changes from RFC v1
(https://lore.kernel.org/[email protected])
- Consider PAGE_SIZE compression successes as failures.
- Use zpool for storing incompressible pages.
- Test with zswap shrinker enabled.
- Wordsmith changelog and comments.
- Add documentation of save_incompressible_pages parameter.

Quoted 2 lines
 mm/zswap.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)

I'll let Johannes chime in as well, but LGTM.

Thanks!

#2 lore raw
In reply to SeongJae Park

Hi SeongJae,

Thanks for doing that. I am working on something related to this as well. Please consider CC me on your next version.

Quoted 15 lines · SeongJae Park

When zswap writeback is enabled and it fails compressing a given page, the page is swapped out to the backing swap device. This behavior breaks the zswap's writeback LRU order, and hence users can experience unexpected latency spikes. If the page is compressed without failure, but results in a size of PAGE_SIZE, the LRU order is kept, but the decompression overhead for loading the page back on the later access is unnecessary.

Keep the LRU order and optimize unnecessary decompression overheads in those cases, by storing the original content as-is in zpool. The length field of zswap_entry will be set appropriately, as PAGE_SIZE, Hence whether it is saved as-is or not (whether decompression is unnecessary) is identified by 'zswap_entry->length == PAGE_SIZE'.

Quoted 3 lines

Because the uncompressed data is saved in zpool, same to the compressed ones, this introduces no change in terms of memory management including movability and migratability of involved pages.

If you store uncompressed data in the zpool, zpool has metadata overhead, e.g. allocating the entry->handle for uncompressed pages. If the page is not compressed, another idea is just skip the zpool, store it as a page in the zswap entry as page. We can make a union of entry->handle and entry->incompressble_page. If entry->length == PAGE_SIZE, use entry->incompressable_page as a page.

The pros is that, on the page fault, there is no need to allocate a new page. You can just move the uncompressed_page into the swap_cache. You save one memcpy on the page fault as well. That will make the incompressible page fault behave very similar to the minor page fault.

The cons is that, now zpool does not account for uncompressed pages, on the second thought, that can be a con as well, the page is not compressed, why should it account as compressed pages?

I know Hugh has some idea to store incompressible pages in the swap cache as well. Hugh?

Quoted 8 lines

This change is also not increasing per zswap entry metadata overhead. But as the number of incompressible pages increases, total zswap metadata overhead is proportionally increased. The overhead should not be problematic in usual cases, since the zswap metadata for single zswap entry is much smaller than PAGE_SIZE, and in common zswap use cases there should be a sufficient amount of compressible pages. Also it can be mitigated by the zswap writeback.

Quoted 8 lines

When the writeback is disabled, the additional overhead could be problematic. For the case, keep the current behavior that just returns the failure and let swap_writeout() put the page back to the active LRU list in the case.

Knowing how many compression failures happened will be useful for future investigations. investigations. Add a new debugfs file, compress_fail, for the purpose.

Again, I think we should distinguish the crypto engine fail vs ration failure.

Quoted 161 lines

Tests
-----

I tested this patch using a simple self-written microbenchmark that is available at GitHub[1]. You can reproduce the test I did by executing run_tests.sh of the repo on your system. Note that the repo's documentation is not good as of this writing, so you may need to read and use the code.

The basic test scenario is simple. Run a test program making artificial accesses to memory having artificial content under memory.high-set memory limit and measure how many accesses were made in given time.

The test program repeatedly and randomly access three anonymous memory regions. The regions are all 500 MiB size, and accessed in the same probability. Two of those are filled up with a simple content that can easily be compressed, while the remaining one is filled up with a content that read from /dev/urandom, which is easy to fail at compressing to <PAGE_SIZE size. The program runs for two minutes and prints out the number of accesses made every five seconds.

The test script runs the program under below seven configurations.

- 0: memory.high is set to 2 GiB, zswap is disabled.
- 1-1: memory.high is set to 1350 MiB, zswap is disabled.
- 1-2: On 1-1, zswap is enabled without this patch.
- 1-3: On 1-2, this patch is applied.

For all zswap enabled cases, zswap shrinker is enabled.

Configuration '0' is for showing the original memory performance. Configurations 1-1, 1-2 and 1-3 are for showing the performance of swap, zswap, and this patch under a level of memory pressure (~10% of working set).

Because the per-5 seconds performance is not very reliable, I measured the average of that for the last one minute period of the test program run. I also measured a few vmstat counters including zswpin, zswpout, zswpwb, pswpin and pswpout during the test runs.

The measurement results are as below. To save space, I show performance numbers that are normalized to that of the configuration '0' (no memory pressure), only. The averaged accesses per 5 seconds of configuration '0' was 36493417.75.

    config            0       1-1     1-2      1-3
    perf_normalized   1.0000  0.0057  0.0235   0.0367
    perf_stdev_ratio  0.0582  0.0652  0.0167   0.0346
    zswpin            0       0       3548424  1999335
    zswpout           0       0       3588817  2361689
    zswpwb            0       0       10214    340270
    pswpin            0       485806  772038   340967
    pswpout           0       649543  144773   340270

'perf_normalized' is the performance metric, normalized to that of configuration '0' (no pressure). 'perf_stdev_ratio' is the standard deviation of the averaged data points, as a ratio to the averaged metric

value.  For example, configuration '0' performance was showing 5.8%
stdev.  Configurations 1-1 and 1-3 were having about 6.5% and 6.1%
stdev.  Also the results were highly variable between multiple runs.  So

this result is not very stable but just showing ball park figures. Please keep this in your mind when reading these results.

Under about 10% of working set memory pressure, the performance was dropped to about 0.57% of no-pressure one, when the normal swap is used (1-1). Note that ~10% working set pressure is already extreme, at least on this test setup. No one would desire system setups that can degrade performance to 0.57% of the best case.

By turning zswap on (1-2), the performance was improved about 4x, resulting in about 2.35% of no-pressure one. Because of the incompressible pages in the third memory region, a significant amount of (non-zswap) swap I/O operations were made, though.

By applying this patch (1-3), about 56% performance improvement was made, resulting in about 3.67% of no-pressure one. Reduced pswpin of 1-3 compared to 1-2 let us see where this improvement came from.

Related Works
-------------

This is not an entirely new attempt. Nhat Pham and Takero Funaki tried very similar approaches in October 2023[2] and April 2024[3], respectively. The two approaches didn't get merged mainly due to the metadata overhead concern. I described why I think that shouldn't be a problem for this change, which is automatically disabled when writeback is disabled, at the beginning of this changelog.

This patch is not particularly different from those, and actually built upon those. I wrote this from scratch again, though. Hence adding Suggested-by tags for them. Actually Nhat first suggested this to me offlist.

Historically, writeback disabling was introduced partially as a way to solve the LRU order issue. Yosry pointed out[4] this is still suboptimal when the incompressible pages are cold, since the incompressible pages will continuously be tried to be zswapped out, and burn CPU cycles for compression attempts that will anyway fail. One imaginable solution for the problem is reusing the swapped-out page and its struct page to store in the zswap pool. But that's out of the scope of this patch.

[1] https://github.com/sjp38/eval_zswap/blob/master/run.sh
[2] https://lore.kernel.org/[email protected]
[3] https://lore.kernel.org/[email protected]
[4] https://lore.kernel.org/CAJD7tkZXS-UJVAFfvxJ0nNgTzWBiqepPYA4hEozi01_qktkitg@mail.gmail.com

---
Changes from v1
(https://lore.kernel.org/[email protected])
- Optimize out memcpy() per incompressible page saving, using k[un]map_local().
- Add a debugfs file for counting compression failures.
- Use a clear form of a ternary operation.
- Add the history of writeback disabling with a link.
- Wordsmith comments.

Changes from RFC v2
(https://lore.kernel.org/[email protected])
- Fix race conditions at decompressed pages identification.
- Remove the parameter and make saving as-is the default behavior.
- Open-code main changes.
- Clarify there is no memory management changes on the cover letter.
- Remove 20% pressure case from test results, since it is arguably too extreme and only adds confusion.
- Drop RFC tag.

Changes from RFC v1
(https://lore.kernel.org/[email protected])
- Consider PAGE_SIZE compression successes as failures.
- Use zpool for storing incompressible pages.
- Test with zswap shrinker enabled.
- Wordsmith changelog and comments.
- Add documentation of save_incompressible_pages parameter.

 mm/zswap.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)
mm/zswap.c +2 -2
index 3c0fd8a13718..0fb940d03268 100644
@@ -60,6 +60,8 @@ static u64 zswap_written_back_pages;
 static u64 zswap_reject_reclaim_fail;
 /* Store failed due to compression algorithm failure */
 static u64 zswap_reject_compress_fail;
+/* Compression into a size of <PAGE_SIZE failed */
+static u64 zswap_compress_fail;
 /* Compressed page was too big for the allocator to (optimally) store */
 static u64 zswap_reject_compress_poor;
 /* Load or writeback failed due to decompression failure */
@@ -976,8 +978,26 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
         */
        comp_ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait);
        dlen = acomp_ctx->req->dlen;
-       if (comp_ret)
-               goto unlock;

mm/zswap.c static bool zswap_compress(struct page *page, struct zswap_entry *entry, · line 990

Quoted 10 lines
mm/zswap.c +10 -0
+
+       /*
+        * If a page cannot be compressed into a size smaller than PAGE_SIZE,
+        * save the content as is without a compression, to keep the LRU order
+        * of writebacks.  If writeback is disabled, reject the page since it
+        * only adds metadata overhead.  swap_writeout() will put the page back
+        * to the active LRU list in the case.
+        */
+       if (comp_ret || dlen >= PAGE_SIZE) {
+               zswap_compress_fail++;

I feel that mixing comp_ret and dlen size if tested here complicates the nested if statement and the behavior as well.
One behavior change is that, if the comp_ret != 0, it means the compression crypt engine has some error. e.g. have some bad chip always fail the compression. That is a different error case than the compression was successful and the compression rate is not good. In this case the patch makes the page store in zpool for cryto engine failure, which does not help.

Is there any reason you want to store the page in zpool when the compression engine (not the ratio) fails?

What do you say about the following alternative, this keeps the original behavior if compression engines fail.

     if (comp_ret) {
          zswap_compress_egine_fail++;
          goto unlock;
     }

     if (dlen >= PAGE_SIZE) {
        zswap_compress_fail++;
        if (!mem_cgroup_zswap_writeback_enabled(
                                      folio_memcg(page_folio(page)))) {
              comp_ret = -EINVAL;
              goto unlock;
        }
       dlen = PAGE_SIZE;
       dst = kmap_local_page(page);
    }

Overall I feel this alternative is simpler and easier to read. I can be biased towards my own code :-).
I think we should treat the compression engine failure separately from the compression rate failure. The engine failure is rare and we should know about it as a real error. The compression rate failure is normal.

Chris

Quoted 49 lines
mm/zswap.c +20 -0
+                       comp_ret = 0;
+                       dlen = PAGE_SIZE;
+                       dst = kmap_local_page(page);
+               } else {
+                       comp_ret = comp_ret ? comp_ret : -EINVAL;
+                       goto unlock;
+               }
+       }
 
        zpool = pool->zpool;
        gfp = GFP_NOWAIT | __GFP_NORETRY | __GFP_HIGHMEM | __GFP_MOVABLE;
@@ -990,6 +1010,8 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
        entry->length = dlen;
 
 unlock:
+       if (dst != acomp_ctx->buffer)
+               kunmap_local(dst);
        if (comp_ret == -ENOSPC || alloc_ret == -ENOSPC)
                zswap_reject_compress_poor++;
        else if (comp_ret)
@@ -1012,6 +1034,14 @@ static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio)
        acomp_ctx = acomp_ctx_get_cpu_lock(entry->pool);
        obj = zpool_obj_read_begin(zpool, entry->handle, acomp_ctx->buffer);
 
+       /* zswap entries of length PAGE_SIZE are not compressed. */
+       if (entry->length == PAGE_SIZE) {
+               memcpy_to_folio(folio, 0, obj, entry->length);
+               zpool_obj_read_end(zpool, entry->handle, obj);
+               acomp_ctx_put_unlock(acomp_ctx);
+               return true;
+       }
+
        /*
         * zpool_obj_read_begin() might return a kmap address of highmem when
         * acomp_ctx->buffer is not used.  However, sg_init_one() does not
@@ -1809,6 +1839,8 @@ static int zswap_debugfs_init(void)
                           zswap_debugfs_root, &zswap_reject_kmemcache_fail);
        debugfs_create_u64("reject_compress_fail", 0444,
                           zswap_debugfs_root, &zswap_reject_compress_fail);
+       debugfs_create_u64("compress_fail", 0444,
+                          zswap_debugfs_root, &zswap_compress_fail);
        debugfs_create_u64("reject_compress_poor", 0444,
                           zswap_debugfs_root, &zswap_reject_compress_poor);
        debugfs_create_u64("decompress_fail", 0444,

base-commit: 44fa6646d975349f6499d1aeb0ed826680d0bb5c --
2.39.5

#3 lore raw

Hi Chris,

Quoted 4 lines · On Wed, 13 Aug 2025 10:07:18 -0700 Chris Li <[email protected]> wrote:

Hi SeongJae,

Thanks for doing that. I am working on something related to this as well. Please consider CC me on your next version.

Thank you for chiming in, I will do so.

Quoted 21 lines · SeongJae Park
Quoted 15 lines

When zswap writeback is enabled and it fails compressing a given page, the page is swapped out to the backing swap device. This behavior breaks the zswap's writeback LRU order, and hence users can experience unexpected latency spikes. If the page is compressed without failure, but results in a size of PAGE_SIZE, the LRU order is kept, but the decompression overhead for loading the page back on the later access is unnecessary.

Keep the LRU order and optimize unnecessary decompression overheads in those cases, by storing the original content as-is in zpool. The length field of zswap_entry will be set appropriately, as PAGE_SIZE, Hence whether it is saved as-is or not (whether decompression is unnecessary) is identified by 'zswap_entry->length = PAGE_SIZE'.

Quoted 3 lines

Because the uncompressed data is saved in zpool, same to the compressed ones, this introduces no change in terms of memory management including movability and migratability of involved pages.

Quoted 6 lines

If you store uncompressed data in the zpool, zpool has metadata overhead, e.g. allocating the entry->handle for uncompressed pages. If the page is not compressed, another idea is just skip the zpool, store it as a page in the zswap entry as page. We can make a union of entry->handle and entry->incompressble_page. If entry->length = PAGE_SIZE, use entry->incompressable_page as a page.

My RFC v1[1] was using a kind of such approach. I changed it to use zpool starting from RFC v2, following the suggestions from Nhat, Johannes and Joshua. It was suggested to use zpool to reuse its support of migration, highmem handling, and stats. And I agree their suggestions. David also raised a question on RFC v2, about the movability behavior changes[2].

I agree we will get more metadata overhead. Nonetheless, I argue it is not a big problem and can be mitigated in writeback-enabled environments. Hence this feature is disabled on writeback-disabled case. Next paragraph on the original cover letter explains my points about this.

Quoted 5 lines

The pros is that, on the page fault, there is no need to allocate a new page. You can just move the uncompressed_page into the swap_cache. You save one memcpy on the page fault as well. That will make the incompressible page fault behave very similar to the minor page fault.

I agree this can save one memcpy, and that's cool idea! Nonetheless, this patch is not making the [z]swap-in overhead worse. Rather, this patch also slightly improve it for incompressible pages case by skipping the decompression. Also, if we take this way, I think we should also need to make a good answer to the zswapped-page migrations etc.

So, if we agree on my justification about the metadata overhead, I think this could be done as a followup work of this patch?

Quoted 4 lines

The cons is that, now zpool does not account for uncompressed pages, on the second thought, that can be a con as well, the page is not compressed, why should it account as compressed pages?

I agree it might look weird. But, in my humble opinion, in a perspective of thinking it as zswap backend storage, and by thinking the definition of compression in a flexible way, this may be understandable and not cause real user problems?

Quoted 17 lines

I know Hugh has some idea to store incompressible pages in the swap cache as well. Hugh?

Quoted 13 lines

This change is also not increasing per zswap entry metadata overhead. But as the number of incompressible pages increases, total zswap metadata overhead is proportionally increased. The overhead should not be problematic in usual cases, since the zswap metadata for single zswap entry is much smaller than PAGE_SIZE, and in common zswap use cases there should be a sufficient amount of compressible pages. Also it can be mitigated by the zswap writeback.

When the writeback is disabled, the additional overhead could be problematic. For the case, keep the current behavior that just returns the failure and let swap_writeout() put the page back to the active LRU list in the case.

Quoted 5 lines
Quoted 3 lines

Knowing how many compression failures happened will be useful for future investigations. investigations. Add a new debugfs file, compress_fail, for the purpose.

Again, I think we should distinguish the crypto engine fail vs ration failure.

I see you left more detailed good opinions about this below. I will reply there.

[...]

Quoted 33 lines
Quoted 22 lines
 mm/zswap.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)
mm/zswap.c +2 -2
index 3c0fd8a13718..0fb940d03268 100644
@@ -60,6 +60,8 @@ static u64 zswap_written_back_pages;
 static u64 zswap_reject_reclaim_fail;
 /* Store failed due to compression algorithm failure */
 static u64 zswap_reject_compress_fail;
+/* Compression into a size of <PAGE_SIZE failed */
+static u64 zswap_compress_fail;
 /* Compressed page was too big for the allocator to (optimally) store */
 static u64 zswap_reject_compress_poor;
 /* Load or writeback failed due to decompression failure */
@@ -976,8 +978,26 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
         */
        comp_ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait);
        dlen = acomp_ctx->req->dlen;
-       if (comp_ret)
-               goto unlock;

mm/zswap.c static bool zswap_compress(struct page *page, struct zswap_entry *entry, · line 990

Quoted 10 lines
mm/zswap.c +10 -0
+
+       /*
+        * If a page cannot be compressed into a size smaller than PAGE_SIZE,
+        * save the content as is without a compression, to keep the LRU order
+        * of writebacks.  If writeback is disabled, reject the page since it
+        * only adds metadata overhead.  swap_writeout() will put the page back
+        * to the active LRU list in the case.
+        */
+       if (comp_ret || dlen >= PAGE_SIZE) {
+               zswap_compress_fail++;
Quoted 8 lines

I feel that mixing comp_ret and dlen size if tested here complicates the nested if statement and the behavior as well.
One behavior change is that, if the comp_ret != 0, it means the compression crypt engine has some error. e.g. have some bad chip always fail the compression. That is a different error case than the compression was successful and the compression rate is not good. In this case the patch makes the page store in zpool for cryto engine failure, which does not help.

I agree the code has rooms to improve, in terms of the readability, and keeping fine grained observailibty.

Quoted 3 lines

Is there any reason you want to store the page in zpool when the compression engine (not the ratio) fails?

The main problem this patch tries to solve is the LRU order corruption. In any case, I want to keep the order if possible.

Quoted 6 lines

What do you say about the following alternative, this keeps the original behavior if compression engines fail.

     if (comp_ret) {
          zswap_compress_egine_fail++;

I agree this counter can be useful.

Quoted 1 line
          goto unlock;

This will make the page go directly to underlying slower swap device, and hence cause LRU order inversion. So unfortuately I have to say this is not the behavior I want.

I agree engine failure is not frequent, so this behavior might not really make problem to me. But, still I don't see a reason to let the page go directly to swap and make LRU order unexpected.

Quoted 4 lines
     }

     if (dlen >= PAGE_SIZE) {
        zswap_compress_fail++;

I define compress failure here as a failure of attempt to compress the given page's content into a size smaller than PAGE_SIZE. It is a superset including both "ratio" failure and "engine" failure. Hence I think zswap_compress_fail should be increased even in the upper case.

We can discuss about re-defining and re-naming what each stat means, of course, if you want. But I think even if we keep the current definitions and names, we can still get the ratio failures if we add your nicely suggested zswap_compress_engine_fail, as
'zswap_compress_fail - zswap_compress_engine_fail', so we might not really need re-definition and re-naming?

Quoted 10 lines
        if (!mem_cgroup_zswap_writeback_enabled(
                                      folio_memcg(page_folio(page)))) {
              comp_ret = -EINVAL;
              goto unlock;
        }
       dlen = PAGE_SIZE;
       dst = kmap_local_page(page);
    }

Overall I feel this alternative is simpler and easier to read.

I agree this code is nice to read. Nonetheless I have to say the behavior is somewhat different from what I want.

Quoted 5 lines

I can
be biased towards my own code :-).
I think we should treat the compression engine failure separately from the compression rate failure. The engine failure is rare and we should know about it as a real error. The compression rate failure is normal.

Again, I agree having the observability would be nice. I can add a new counter for that, like below attached patch, for example.

[1] https://lore.kernel.org/[email protected]
[2] https://lore.kernel.org/[email protected]

Thanks,
SJ

[...]

==== >8 ====
From 839459619cf2c1a66d0a82361808bab499feefda Mon Sep 17 00:00:00 2001
From: SeongJae Park <[email protected]>
Date: Wed, 13 Aug 2025 11:13:24 -0700
Subject: [PATCH] mm/zswap: add crypto engine failures debugfs counter

compress_fail counts failures of general attempts to compress a given apge into a size smaller than PAGE_SIZE. It could be happened because the content of the page is not easy to be compressed, or the crypto engine error-ed for any reason. Adda new debugfs file, namely compress_engine_fail, for coutning the latter case. Keep the comress_fail count, since the failures due to the compression ratio failure can be calculated by subtracting the new counter (compress_engine_fail) value from that of the existing one (compress_fail).

---
 mm/zswap.c | 6 ++++++
 1 file changed, 6 insertions(+)
mm/zswap.c +6 -0
index 0fb940d03268..6b2144126fba 100644
@@ -62,6 +62,8 @@ static u64 zswap_reject_reclaim_fail;
 static u64 zswap_reject_compress_fail;
 /* Compression into a size of <PAGE_SIZE failed */
 static u64 zswap_compress_fail;
+/* Compression failed by the crypto engine */
+static u64 zswap_compress_engine_fail;
 /* Compressed page was too big for the allocator to (optimally) store */
 static u64 zswap_reject_compress_poor;
 /* Load or writeback failed due to decompression failure */
@@ -988,6 +990,8 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
 	 */
 	if (comp_ret || dlen >= PAGE_SIZE) {
 		zswap_compress_fail++;
+		if (comp_ret)
+			zswap_compress_engine_fail++;
 		if (mem_cgroup_zswap_writeback_enabled(
 					folio_memcg(page_folio(page)))) {
 			comp_ret = 0;
@@ -1841,6 +1845,8 @@ static int zswap_debugfs_init(void)
 			   zswap_debugfs_root, &zswap_reject_compress_fail);
 	debugfs_create_u64("compress_fail", 0444,
 			   zswap_debugfs_root, &zswap_compress_fail);
+	debugfs_create_u64("compress_engine_fail", 0444,
+			   zswap_debugfs_root, &zswap_compress_engine_fail);
 	debugfs_create_u64("reject_compress_poor", 0444,
 			   zswap_debugfs_root, &zswap_reject_compress_poor);
 	debugfs_create_u64("decompress_fail", 0444,
Signature · 1 line
2.39.5
#4 lore raw
Quoted 6 lines · SeongJae Park

My RFC v1[1] was using a kind of such approach. I changed it to use zpool starting from RFC v2, following the suggestions from Nhat, Johannes and Joshua. It was suggested to use zpool to reuse its support of migration, highmem handling, and stats. And I agree their suggestions. David also raised a question on RFC v2, about the movability behavior changes[2].

Thanks for the pointer, that is an interesting read.

Quoted 1 line

I agree we will get more metadata overhead. Nonetheless, I argue it is not a

Keep in mind that is more than just metadata. There is another hidden overhead of using zpool to store your full page compared to directly using the page. When the page is read, because most likely the zpool back end is not at a position aligned to the page. That zpool page will not be able to free immediately, those fragmented zpool will need to wait for compaction to free. it.

Quoted 5 lines

big problem and can be mitigated in writeback-enabled environments. Hence this feature is disabled on writeback-disabled case. Next paragraph on the original cover letter explains my points about this.

Quoted 1 line
Quoted 7 lines
Quoted 4 lines

The pros is that, on the page fault, there is no need to allocate a new page. You can just move the uncompressed_page into the swap_cache. You save one memcpy on the page fault as well. That will make the incompressible page fault behave very similar to the minor page fault.

I agree this can save one memcpy, and that's cool idea! Nonetheless, this patch is not making the [z]swap-in overhead worse. Rather, this patch also

We might still wait to evaluate the lost/gain vs store the incompressible in swap cache. Google actually has an internal patch to store the incompressible pages in swap cache and move them out of the LRU to another already existing list. I can clean it up a bit and send it to the list for comparison. This approach will not mess with the zswap LRU because the incompressible data is not moving into zswap. There is no page fault to handle the incompressible pages.

Quoted 3 lines

slightly improve it for incompressible pages case by skipping the decompression. Also, if we take this way, I think we should also need to make a good answer to the zswapped-page migrations etc.

Yes, if we keep the store page in the zswap approach, we might have to optimize inside zsmalloc to handle full page storing better.

Quoted 2 lines

So, if we agree on my justification about the metadata overhead, I think this could be done as a followup work of this patch?

Sure. I am most interested in getting the best overall solution. No objects to get it now vs later.

Quoted 10 lines
Quoted 5 lines

The cons is that, now zpool does not account for uncompressed pages, on the second thought, that can be a con as well, the page is not compressed, why should it account as compressed pages?

I agree it might look weird. But, in my humble opinion, in a perspective of thinking it as zswap backend storage, and by thinking the definition of compression in a flexible way, this may be understandable and not cause real user problems?

I think the real problem is hiding the real error with non errors like data not compressible. If you want to store uncompressed data in the zswap layer by design, that is fine. Just need to reason the trade off vs other options like store in swap cache etc.

Quoted 47 lines

[...]

Quoted 33 lines
Quoted 22 lines
 mm/zswap.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)
mm/zswap.c +2 -2
index 3c0fd8a13718..0fb940d03268 100644
@@ -60,6 +60,8 @@ static u64 zswap_written_back_pages;
 static u64 zswap_reject_reclaim_fail;
 /* Store failed due to compression algorithm failure */
 static u64 zswap_reject_compress_fail;
+/* Compression into a size of <PAGE_SIZE failed */
+static u64 zswap_compress_fail;
 /* Compressed page was too big for the allocator to (optimally) store */
 static u64 zswap_reject_compress_poor;
 /* Load or writeback failed due to decompression failure */
@@ -976,8 +978,26 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
         */
        comp_ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait);
        dlen = acomp_ctx->req->dlen;
-       if (comp_ret)
-               goto unlock;

mm/zswap.c static bool zswap_compress(struct page *page, struct zswap_entry *entry, · line 990

Quoted 10 lines
mm/zswap.c +10 -0
+
+       /*
+        * If a page cannot be compressed into a size smaller than PAGE_SIZE,
+        * save the content as is without a compression, to keep the LRU order
+        * of writebacks.  If writeback is disabled, reject the page since it
+        * only adds metadata overhead.  swap_writeout() will put the page back
+        * to the active LRU list in the case.
+        */
+       if (comp_ret || dlen >= PAGE_SIZE) {
+               zswap_compress_fail++;
Quoted 8 lines

I feel that mixing comp_ret and dlen size if tested here complicates the nested if statement and the behavior as well.
One behavior change is that, if the comp_ret != 0, it means the compression crypt engine has some error. e.g. have some bad chip always fail the compression. That is a different error case than the compression was successful and the compression rate is not good. In this case the patch makes the page store in zpool for cryto engine failure, which does not help.

I agree the code has rooms to improve, in terms of the readability, and keeping fine grained observailibty.

Quoted 1 line
Quoted 5 lines
Quoted 2 lines

Is there any reason you want to store the page in zpool when the compression engine (not the ratio) fails?

The main problem this patch tries to solve is the LRU order corruption. In any case, I want to keep the order if possible.

In that case, does it mean that in order to keep the LRU, you never want to write from zswap to a real back end device? That will make zswap behave more like a standalone tier.

Quoted 9 lines
Quoted 7 lines

What do you say about the following alternative, this keeps the original behavior if compression engines fail.

     if (comp_ret) {
          zswap_compress_egine_fail++;

I agree this counter can be useful.

Ack.

Quoted 6 lines
Quoted 2 lines
          goto unlock;

This will make the page go directly to underlying slower swap device, and hence cause LRU order inversion. So unfortuately I have to say this is not the behavior I want.

Ack, that is a design choice. I understand now.

Quoted 7 lines

I agree engine failure is not frequent, so this behavior might not really make problem to me. But, still I don't see a reason to let the page go directly to swap and make LRU order unexpected.

Quoted 2 lines
     }
Quoted 7 lines
Quoted 2 lines
     if (dlen >= PAGE_SIZE) {
        zswap_compress_fail++;

I define compress failure here as a failure of attempt to compress the given page's content into a size smaller than PAGE_SIZE. It is a superset including both "ratio" failure and "engine" failure. Hence I think zswap_compress_fail should be increased even in the upper case.

I slept over it a bit. Now I think we should make this a counter of how many uncompressed pages count stored in zswap. Preperbelly as per memcg counter.
I saw that you implement it as a counter in your V1. Does the zsmalloc already track this information in the zspool class? Having this per cgroup information we can evaluate how much zswap is saving. Some jobs like databases might store a lot of hash value and encrypted data which is not compressible. In that case, we might just pass the whole zswap tier as a whole. The overall compression ratio will reflect that as well. Having the separate per memcg counter is kind of useful as well.

Quoted 7 lines

We can discuss about re-defining and re-naming what each stat means, of course, if you want. But I think even if we keep the current definitions and names, we can still get the ratio failures if we add your nicely suggested zswap_compress_engine_fail, as
'zswap_compress_fail - zswap_compress_engine_fail', so we might not really need re-definition and re-naming?

I am actually less interested in the absolute failure number which keeps increasing, more on how much incompressible zswap is stored. That counter + number of engine errors should be perfect.

Quoted 10 lines
Quoted 10 lines
        if (!mem_cgroup_zswap_writeback_enabled(
                                      folio_memcg(page_folio(page)))) {
              comp_ret = -EINVAL;
              goto unlock;
        }
       dlen = PAGE_SIZE;
       dst = kmap_local_page(page);
    }
Quoted 4 lines
Quoted 1 line

Overall I feel this alternative is simpler and easier to read.

I agree this code is nice to read. Nonetheless I have to say the behavior is somewhat different from what I want.

Even if you keep the current behavior, you can move the invert the test condition and then remove the "else + goto" similar to the above. That will make your code less and flatter. I will need to think about whether we can assign the return value less.

Another point I would like to make is that you currently make the cut off threshold as page size. The ideal threshold might be something slightly smaller than page size. The reason is that the zsmalloc has a fixed size chuck to store it. If your page is close to full, it will store the data in the same class as the full page. You are not gaining anything from zsmalloc if the store page compression ratio is 95%. That 5% will be in the waste in the zsmalloc class fragment anyway. So the trade off is, decompress 95% of a page vs memcpy 100% of a page. It is likely memcpy 100% is faster. I don't know the exact ideal cut off of threshold. If I had to guess, I would guess below 90%.

Quoted 9 lines
Quoted 6 lines

I can
be biased towards my own code :-).
I think we should treat the compression engine failure separately from the compression rate failure. The engine failure is rare and we should know about it as a real error. The compression rate failure is normal.

Again, I agree having the observability would be nice. I can add a new counter for that, like below attached patch, for example.

I would love that. Make that per memcg if possible :-)

Quoted 3 lines

[1] https://lore.kernel.org/[email protected]
[2] https://lore.kernel.org/[email protected]

Thanks for the pointer, good read.

Chris

maintainer
#5 lore raw
Quoted 11 lines · Chris Li
Quoted 6 lines · SeongJae Park

My RFC v1[1] was using a kind of such approach. I changed it to use zpool starting from RFC v2, following the suggestions from Nhat, Johannes and Joshua. It was suggested to use zpool to reuse its support of migration, highmem handling, and stats. And I agree their suggestions. David also raised a question on RFC v2, about the movability behavior changes[2].

Thanks for the pointer, that is an interesting read.

Quoted 8 lines
Quoted 1 line

I agree we will get more metadata overhead. Nonetheless, I argue it is not a

Keep in mind that is more than just metadata. There is another hidden overhead of using zpool to store your full page compared to directly using the page. When the page is read, because most likely the zpool back end is not at a position aligned to the page. That zpool page will not be able to free immediately, those fragmented zpool will need to wait for compaction to free. it.

It's been awhile since I last worked on zsmalloc, but IIRC zsmalloc handles these pages PAGE_SIZE sized object by just handing out a full page. I skimmed through zsmalloc code, and it seems like for this size_class, each zspage just contain one base page?

(check out the logic around ZsHugePage and calculate_zspage_chain_size() in mm/zsmalloc.c. Confusing name, I know).

So we won't need compaction to get rid of these buffers for incompressible pages, at free time. There might still be some extra overhead (metadata), but at least I assume we can free these pages easily?

Quoted 14 lines
Quoted 6 lines

big problem and can be mitigated in writeback-enabled environments. Hence this feature is disabled on writeback-disabled case. Next paragraph on the original cover letter explains my points about this.

Quoted 1 line
Quoted 7 lines
Quoted 4 lines

The pros is that, on the page fault, there is no need to allocate a new page. You can just move the uncompressed_page into the swap_cache. You save one memcpy on the page fault as well. That will make the incompressible page fault behave very similar to the minor page fault.

I agree this can save one memcpy, and that's cool idea! Nonetheless, this patch is not making the [z]swap-in overhead worse. Rather, this patch also

Quoted 7 lines

We might still wait to evaluate the lost/gain vs store the incompressible in swap cache. Google actually has an internal patch to store the incompressible pages in swap cache and move them out of the LRU to another already existing list. I can clean it up a bit and send it to the list for comparison. This approach will not mess with the zswap LRU because the incompressible data is not moving into zswap. There is no page fault to handle the incompressible pages.

We can always iterate to get the best of all worlds :) One objective at a time.

BTW, May I ask - how/when do we move the incompressible pages out of the "incompressible LRU"? I believe there needs to be some sort of scanning mechanism to detect dirtying right?

Quoted 7 lines
Quoted 4 lines

slightly improve it for incompressible pages case by skipping the decompression. Also, if we take this way, I think we should also need to make a good answer to the zswapped-page migrations etc.

Yes, if we keep the store page in the zswap approach, we might have to optimize inside zsmalloc to handle full page storing better.

I believe it already does - check my response above. But I can be wrong.

Quoted 6 lines
Quoted 3 lines

So, if we agree on my justification about the metadata overhead, I think this could be done as a followup work of this patch?

Sure. I am most interested in getting the best overall solution. No objects to get it now vs later.

Agree. We can always iterate on solutions. No big deal.

Quoted 105 lines
Quoted 11 lines
Quoted 6 lines

The cons is that, now zpool does not account for uncompressed pages, on the second thought, that can be a con as well, the page is not compressed, why should it account as compressed pages?

I agree it might look weird. But, in my humble opinion, in a perspective of thinking it as zswap backend storage, and by thinking the definition of compression in a flexible way, this may be understandable and not cause real user problems?

I think the real problem is hiding the real error with non errors like data not compressible. If you want to store uncompressed data in the zswap layer by design, that is fine. Just need to reason the trade off vs other options like store in swap cache etc.

Quoted 47 lines

[...]

Quoted 33 lines
Quoted 22 lines
 mm/zswap.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)
mm/zswap.c +2 -2
index 3c0fd8a13718..0fb940d03268 100644
@@ -60,6 +60,8 @@ static u64 zswap_written_back_pages;
 static u64 zswap_reject_reclaim_fail;
 /* Store failed due to compression algorithm failure */
 static u64 zswap_reject_compress_fail;
+/* Compression into a size of <PAGE_SIZE failed */
+static u64 zswap_compress_fail;
 /* Compressed page was too big for the allocator to (optimally) store */
 static u64 zswap_reject_compress_poor;
 /* Load or writeback failed due to decompression failure */
@@ -976,8 +978,26 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
         */
        comp_ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait);
        dlen = acomp_ctx->req->dlen;
-       if (comp_ret)
-               goto unlock;

mm/zswap.c static bool zswap_compress(struct page *page, struct zswap_entry *entry, · line 990

Quoted 10 lines
mm/zswap.c +10 -0
+
+       /*
+        * If a page cannot be compressed into a size smaller than PAGE_SIZE,
+        * save the content as is without a compression, to keep the LRU order
+        * of writebacks.  If writeback is disabled, reject the page since it
+        * only adds metadata overhead.  swap_writeout() will put the page back
+        * to the active LRU list in the case.
+        */
+       if (comp_ret || dlen >= PAGE_SIZE) {
+               zswap_compress_fail++;
Quoted 8 lines

I feel that mixing comp_ret and dlen size if tested here complicates the nested if statement and the behavior as well.
One behavior change is that, if the comp_ret != 0, it means the compression crypt engine has some error. e.g. have some bad chip always fail the compression. That is a different error case than the compression was successful and the compression rate is not good. In this case the patch makes the page store in zpool for cryto engine failure, which does not help.

I agree the code has rooms to improve, in terms of the readability, and keeping fine grained observailibty.

Quoted 1 line
Quoted 5 lines
Quoted 2 lines

Is there any reason you want to store the page in zpool when the compression engine (not the ratio) fails?

The main problem this patch tries to solve is the LRU order corruption. In any case, I want to keep the order if possible.

In that case, does it mean that in order to keep the LRU, you never want to write from zswap to a real back end device? That will make zswap behave more like a standalone tier.

Quoted 9 lines
Quoted 7 lines

What do you say about the following alternative, this keeps the original behavior if compression engines fail.

     if (comp_ret) {
          zswap_compress_egine_fail++;

I agree this counter can be useful.

Ack.

Quoted 6 lines
Quoted 2 lines
          goto unlock;

This will make the page go directly to underlying slower swap device, and hence cause LRU order inversion. So unfortuately I have to say this is not the behavior I want.

Ack, that is a design choice. I understand now.

Quoted 10 lines

I agree engine failure is not frequent, so this behavior might not really make problem to me. But, still I don't see a reason to let the page go directly to swap and make LRU order unexpected.

Quoted 4 lines
     }

     if (dlen >= PAGE_SIZE) {
        zswap_compress_fail++;
Quoted 8 lines
Quoted 4 lines

I define compress failure here as a failure of attempt to compress the given page's content into a size smaller than PAGE_SIZE. It is a superset includin. both "ratio" failure and "engine" failure. Hence I think zswap_compress_fail should be increased even in the upper case.

I slept over it a bit. Now I think we should make this a counter of how many uncompressed pages count stored in zswap. Preperbelly as per memcg counter.

Actually, yeah I asked about this counter in a review in an earlier version as well, then I completely forgot about it :)

Quoted 2 lines

I saw that you implement it as a counter in your V1. Does the zsmalloc already track this information in the zspool class? Having this per

Kinda sorta. If we build the kernel with CONFIG_ZSMALLOC_STAT, we can get the number of objects in each size_class.

Each time we read, I believe we have to read every size class though. So it's kinda annoying. Whereas here, we can just read an atomic counter? :)

Quoted 3 lines

cgroup information we can evaluate how much zswap is saving. Some jobs like databases might store a lot of hash value and encrypted data which is not compressible. In that case, we might just pass the whole

Hmmm actually, this might be a good point. Lemme think about it a bit more.

(but again, worst case scenario, we can send a follow up patch to add these counters).

Quoted 66 lines

zswap tier as a whole. The overall compression ratio will reflect that as well. Having the separate per memcg counter is kind of useful as well.

Quoted 7 lines

We can discuss about re-defining and re-naming what each stat means, of course, if you want. But I think even if we keep the current definitions and names, we can still get the ratio failures if we add your nicely suggested zswap_compress_engine_fail, as
'zswap_compress_fail - zswap_compress_engine_fail', so we might not really need re-definition and re-naming?

I am actually less interested in the absolute failure number which keeps increasing, more on how much incompressible zswap is stored. That counter + number of engine errors should be perfect.

Quoted 10 lines
Quoted 10 lines
        if (!mem_cgroup_zswap_writeback_enabled(
                                      folio_memcg(page_folio(page)))) {
              comp_ret = -EINVAL;
              goto unlock;
        }
       dlen = PAGE_SIZE;
       dst = kmap_local_page(page);
    }
Quoted 4 lines
Quoted 1 line

Overall I feel this alternative is simpler and easier to read.

I agree this code is nice to read. Nonetheless I have to say the behavior is somewhat different from what I want.

Even if you keep the current behavior, you can move the invert the test condition and then remove the "else + goto" similar to the above. That will make your code less and flatter. I will need to think about whether we can assign the return value less.

Another point I would like to make is that you currently make the cut off threshold as page size. The ideal threshold might be something slightly smaller than page size. The reason is that the zsmalloc has a fixed size chuck to store it. If your page is close to full, it will store the data in the same class as the full page. You are not gaining anything from zsmalloc if the store page compression ratio is 95%. That 5% will be in the waste in the zsmalloc class fragment anyway. So the trade off is, decompress 95% of a page vs memcpy 100% of a page. It is likely memcpy 100% is faster. I don't know the exact ideal cut off of threshold. If I had to guess, I would guess below 90%.

Quoted 9 lines
Quoted 6 lines

I can
be biased towards my own code :-).
I think we should treat the compression engine failure separately from the compression rate failure. The engine failure is rare and we should know about it as a real error. The compression rate failure is normal.

Again, I agree having the observability would be nice. I can add a new counter for that, like below attached patch, for example.

I would love that. Make that per memcg if possible :-)

Quoted 3 lines

[1] https://lore.kernel.org/[email protected]
[2] https://lore.kernel.org/[email protected]

Thanks for the pointer, good read.

Chris

#6 lore raw
Quoted 3 lines · On Fri, 15 Aug 2025 16:08:50 -0700 Nhat Pham <[email protected]> wrote:
Quoted 2 lines · Chris Li

On Wed, Aug 13, 2025 at 11:20 AM SeongJae Park <[email protected]> wrote:

[...]

Quoted 11 lines
Quoted 3 lines

I slept over it a bit. Now I think we should make this a counter of how many uncompressed pages count stored in zswap. Preperbelly as per memcg counter.

Actually, yeah I asked about this counter in a review in an earlier version as well, then I completely forgot about it :)

Quoted 2 lines

I saw that you implement it as a counter in your V1. Does the zsmalloc already track this information in the zspool class? Having this per

Quoted 6 lines

Kinda sorta. If we build the kernel with CONFIG_ZSMALLOC_STAT, we can get the number of objects in each size_class.

Each time we read, I believe we have to read every size class though. So it's kinda annoying. Whereas here, we can just read an atomic counter? :)

Sounds good. So in the next version (v4), I will drop compress_fail. Instead, I will add two new counters, namely compress_engine_fail and the new atomic counter, say, stored_uncompressed_pages. Please suggest better names or correct me if I'm missing some of your points.

Thanks,
SJ

[...]

#7 lore raw
Quoted 5 lines · SeongJae Park

Sounds good. So in the next version (v4), I will drop compress_fail. Instead, I will add two new counters, namely compress_engine_fail and the new atomic counter, say, stored_uncompressed_pages. Please suggest better names or correct me if I'm missing some of your points.

Sounds great. As I suggest in my other email, if up to me,

"crypto_compress_fail" and "stored_incompressible_pages"

Thanks!

Chris

#8 lore raw
Quoted 10 lines · On Fri, 15 Aug 2025 19:23:10 -0700 Chris Li <[email protected]> wrote:
Quoted 5 lines · SeongJae Park

Sounds good. So in the next version (v4), I will drop compress_fail. Instead, I will add two new counters, namely compress_engine_fail and the new atomic counter, say, stored_uncompressed_pages. Please suggest better names or correct me if I'm missing some of your points.

Sounds great. As I suggest in my other email, if up to me,

"crypto_compress_fail" and "stored_incompressible_pages"

Nice names, thank you! I'll send the next verison soon, following all your suggestions[1] together.

[1] https://lore.kernel.org/CAF8kJuPCpnGAoz=1CBwe46ytpcR0ZUMAEWLFQce7eUWkb+0ERA@mail.gmail.com

Thanks,
SJ

[...]

#9 lore raw
Quoted 10 lines · SeongJae Park
Quoted 7 lines · On Fri, 15 Aug 2025 19:23:10 -0700 Chris Li <[email protected]> wrote:
Quoted 5 lines · SeongJae Park

Sounds good. So in the next version (v4), I will drop compress_fail. Instead, I will add two new counters, namely compress_engine_fail and the new atomic counter, say, stored_uncompressed_pages. Please suggest better names or correct me if I'm missing some of your points.

Quoted 6 lines
Quoted 3 lines

Sounds great. As I suggest in my other email, if up to me,

"crypto_compress_fail" and "stored_incompressible_pages"

Nice names, thank you! I'll send the next verison soon, following all your suggestions[1] together.

Thanks, looking forward to it.

Chris

#10 lore raw
In reply to Nhat Pham
Quoted 9 lines · Nhat Pham
Quoted 8 lines

Keep in mind that is more than just metadata. There is another hidden overhead of using zpool to store your full page compared to directly using the page. When the page is read, because most likely the zpool back end is not at a position aligned to the page. That zpool page will not be able to free immediately, those fragmented zpool will need to wait for compaction to free. it.

Quoted 4 lines

It's been awhile since I last worked on zsmalloc, but IIRC zsmalloc handles these pages PAGE_SIZE sized object by just handing out a full page. I skimmed through zsmalloc code, and it seems like for this size_class, each zspage just contain one base page?

Yes, they have to be full size for the page class that contains the full size. That is exactly what I expected. What I previously mentioned the page might be fragmented and not able to release from zpool is wrong, now you mention it due to the full size. That is also why I make the other point that the ideal cut off threshold can be less than PAGE_SIZE. e.g. after compression 95% of a full page, you might just as well store it as uncompressed. Those 5% will waste in the class margin anyway.

Quoted 9 lines

(check out the logic around ZsHugePage and calculate_zspage_chain_size() in mm/zsmalloc.c. Confusing name, I know).

So we won't need compaction to get rid of these buffers for incompressible pages, at free time. There might still be some extra overhead (metadata), but at least I assume we can free these pages easily?

Maybe, or steal them into the swap cache.

Quoted 8 lines
Quoted 7 lines

We might still wait to evaluate the lost/gain vs store the incompressible in swap cache. Google actually has an internal patch to store the incompressible pages in swap cache and move them out of the LRU to another already existing list. I can clean it up a bit and send it to the list for comparison. This approach will not mess with the zswap LRU because the incompressible data is not moving into zswap. There is no page fault to handle the incompressible pages.

Quoted 13 lines

We can always iterate to get the best of all worlds :) One objective at a time.

BTW, May I ask - how/when do we move the incompressible pages out of the "incompressible LRU"? I believe there needs to be some sort of scanning mechanism to detect dirtying right?

Quoted 7 lines
Quoted 4 lines

slightly improve it for incompressible pages case by skipping the decompression. Also, if we take this way, I think we should also need to make a good answer to the zswapped-page migrations etc.

Yes, if we keep the store page in the zswap approach, we might have to optimize inside zsmalloc to handle full page storing better.

Sure. Ack.

Quoted 10 lines

I believe it already does - check my response above. But I can be wrong.

Quoted 3 lines

I slept over it a bit. Now I think we should make this a counter of how many uncompressed pages count stored in zswap. Preperbelly as per memcg counter.

Actually, yeah I asked about this counter in a review in an earlier version as well, then I completely forgot about it :)

Ack.

Quoted 5 lines
Quoted 4 lines

I saw that you implement it as a counter in your V1. Does the zsmalloc already track this information in the zspool class? Having this per

Quoted 7 lines

Kinda sorta. If we build the kernel with CONFIG_ZSMALLOC_STAT, we can get the number of objects in each size_class.

Each time we read, I believe we have to read every size class though. So it's kinda annoying. Whereas here, we can just read an atomic counter? :)

The data compressible or not is very dependent on the job. A per memcg counter will be more useful than a global overall counter. As a devil's advocate to challenge the per memcg incompressible counter. If most of the data is not compressible, the per memcg overall compression ratio, compressed size vs before compress size can reflect that as well. Depending on how fine grain we want this data.

Quoted 8 lines
Quoted 3 lines

cgroup information we can evaluate how much zswap is saving. Some jobs like databases might store a lot of hash value and encrypted data which is not compressible. In that case, we might just pass the whole

Hmmm actually, this might be a good point. Lemme think about it a bit more.

(but again, worst case scenario, we can send a follow up patch to add these counters).

Agree.

Thanks

Chris

Chris

#11 lore raw
In reply to Chris Li
Quoted 1 line · On Fri, 15 Aug 2025 15:28:59 -0700 Chris Li <[email protected]> wrote:

On Wed, Aug 13, 2025 at 11:20 AM SeongJae Park <[email protected]> wrote:

[...]

Quoted 5 lines

We might still wait to evaluate the lost/gain vs store the incompressible in swap cache. Google actually has an internal patch to store the incompressible pages in swap cache and move them out of the LRU to another already existing list. I can clean it up a bit and send it to the list for comparison.

This would be really nice!

[...]

Quoted 5 lines
Quoted 2 lines

So, if we agree on my justification about the metadata overhead, I think this could be done as a followup work of this patch?

Sure. I am most interested in getting the best overall solution. No objects to get it now vs later.

Thank you for being flexible, Chris. I'm also looking forward to keep collaborating with you on the followup works!

[...]

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Is there any reason you want to store the page in zpool when the compression engine (not the ratio) fails?

The main problem this patch tries to solve is the LRU order corruption. In any case, I want to keep the order if possible.

In that case, does it mean that in order to keep the LRU, you never want to write from zswap to a real back end device?

Not always, but until we have to write back zswapped pages to real back end deevice, and all zswapped pages of lower LRU-order are already wrote back.

[...]

Quoted 11 lines
Quoted 7 lines
Quoted 2 lines
     if (dlen >= PAGE_SIZE) {
        zswap_compress_fail++;

I define compress failure here as a failure of attempt to compress the given page's content into a size smaller than PAGE_SIZE. It is a superset including both "ratio" failure and "engine" failure. Hence I think zswap_compress_fail should be increased even in the upper case.

I slept over it a bit. Now I think we should make this a counter of how many uncompressed pages count stored in zswap. Preperbelly as per memcg counter.

I agree that could be useful. I will add the counter in the next version (v4). But making it for each memcg may be out of the scope of this patch, in my opinion. Would you mind doing per-memcg counter implementation as a followup?

Quoted 1 line

I saw that you implement it as a counter in your V1.

Yes, though it was only for internal usage and therefore not exposed to the user space. I will make it again and expose to the user space via debugfs. Say, stored_uncompressed_pages?

Quoted 2 lines

Does the zsmalloc
already track this information in the zspool class?

zsmalloc provides such information when CONFIG_ZSMALLOC_STAT is enabled, to my understanding.

[...]

Quoted 3 lines

I am actually less interested in the absolute failure number which keeps increasing, more on how much incompressible zswap is stored. That counter + number of engine errors should be perfect.

Sounds good. So the next version (v4) of this patch will provide two new debugfs counters, namely compress_engine_fail, and stored_uncompressed_pages.

[...]

Quoted 7 lines
Quoted 2 lines

I agree this code is nice to read. Nonetheless I have to say the behavior is somewhat different from what I want.

Even if you keep the current behavior, you can move the invert the test condition and then remove the "else + goto" similar to the above. That will make your code less and flatter. I will need to think about whether we can assign the return value less.

Nice idea. What about below?

        if (comp_ret) {
                zswap_compress_engine_fail++;
                dlen = PAGE_SIZE;
        }
        if (dlen >= PAGE_SIZE) {
                zswap_compress_fail++;
                if (!mem_cgroup_zswap_writeback_enabled(
                                        folio_memcg(page_folio(page)))) {
                        comp_ret = -EINVAL;
                        goto unlock;
                }
                comp_ret = 0;
                dlen = PAGE_SIZE;
                dst = kmap_local_page(page);
        }
Quoted 11 lines

Another point I would like to make is that you currently make the cut off threshold as page size. The ideal threshold might be something slightly smaller than page size. The reason is that the zsmalloc has a fixed size chuck to store it. If your page is close to full, it will store the data in the same class as the full page. You are not gaining anything from zsmalloc if the store page compression ratio is 95%. That 5% will be in the waste in the zsmalloc class fragment anyway. So the trade off is, decompress 95% of a page vs memcpy 100% of a page. It is likely memcpy 100% is faster. I don't know the exact ideal cut off of threshold. If I had to guess, I would guess below 90%.

Agreed, this could be another nice followup work to do.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

I can
be biased towards my own code :-).
I think we should treat the compression engine failure separately from the compression rate failure. The engine failure is rare and we should know about it as a real error. The compression rate failure is normal.

Quoted 4 lines
Quoted 2 lines

Again, I agree having the observability would be nice. I can add a new counter for that, like below attached patch, for example.

I would love that. Make that per memcg if possible :-)

As mentioned above, I would like to make only a global counter on debugfs for now, if you don't mind. Let me know if you mind.

Thanks,
SJ

[...]

#12 lore raw
Quoted 7 lines · SeongJae Park
Quoted 2 lines · On Fri, 15 Aug 2025 15:28:59 -0700 Chris Li <[email protected]> wrote:

Sure. I am most interested in getting the best overall solution. No objects to get it now vs later.

Thank you for being flexible, Chris. I'm also looking forward to keep collaborating with you on the followup works!

Likewise.

Quoted 5 lines

[...]

Quoted 3 lines
Quoted 3 lines
Quoted 2 lines

Is there any reason you want to store the page in zpool when the compression engine (not the ratio) fails?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

The main problem this patch tries to solve is the LRU order corruption. In any case, I want to keep the order if possible.

In that case, does it mean that in order to keep the LRU, you never want to write from zswap to a real back end device?

Not always, but until we have to write back zswapped pages to real back end deevice, and all zswapped pages of lower LRU-order are already wrote back.

I see, that makes sense. Only writers to the lower tier while maintaining LRU order.
Thanks for the explanation. I can see the bigger picture now.

Quoted 7 lines
Quoted 3 lines

I slept over it a bit. Now I think we should make this a counter of how many uncompressed pages count stored in zswap. Preperbelly as per memcg counter.

I agree that could be useful. I will add the counter in the next version (v4). But making it for each memcg may be out of the scope of this patch, in my opinion. Would you mind doing per-memcg counter implementation as a followup?

No objections. If the incompressible page count is very bad, it will drag down the overall compression ratio as well. So we do have some per memcg signal to pick it up. The trade off is how complex we want to go for those marginal finer grain measurements.

Quoted 5 lines
Quoted 1 line

I saw that you implement it as a counter in your V1.

Yes, though it was only for internal usage and therefore not exposed to the user space. I will make it again and expose to the user space via debugfs. Say, stored_uncompressed_pages?

I am not very good at naming either. Maybe "stored_incompressible_pages"? Uncompressed pages are the resulting state, it sounds like we choose to store them uncompressed. The root cause is that those pages are incompressible. That is the nature of the page, we don't have a choice to compress them.

Quoted 6 lines
Quoted 3 lines

Does the zsmalloc
already track this information in the zspool class?

zsmalloc provides such information when CONFIG_ZSMALLOC_STAT is enabled, to my understanding.

Thanks. I think that is likely global not per memcg.

Quoted 8 lines

[...]

Quoted 3 lines

I am actually less interested in the absolute failure number which keeps increasing, more on how much incompressible zswap is stored. That counter + number of engine errors should be perfect.

Sounds good. So the next version (v4) of this patch will provide two new debugfs counters, namely compress_engine_fail, and stored_uncompressed_pages.

Sounds good. Maybe "crypto_compress_fail", the engine is the common name as I call it. Here the engine refers to the crypto library so "crypto_compress_fail" matches the code better.

Quoted 5 lines

[...]

Quoted 3 lines
Quoted 2 lines

I agree this code is nice to read. Nonetheless I have to say the behavior is somewhat different from what I want.

Quoted 8 lines
Quoted 4 lines

Even if you keep the current behavior, you can move the invert the test condition and then remove the "else + goto" similar to the above. That will make your code less and flatter. I will need to think about whether we can assign the return value less.

Nice idea. What about below?

        if (comp_ret) {

You can consider adding (comp_ret || !dlen) , because if dlen == 0, something is seriously wrong with the compression, that is a real error.

Quoted 6 lines
                zswap_compress_engine_fail++;
                dlen = PAGE_SIZE;
        }
        if (dlen >= PAGE_SIZE) {
                zswap_compress_fail++;
                if (!mem_cgroup_zswap_writeback_enabled(
Quoted 8 lines
                                        folio_memcg(page_folio(page)))) {
                        comp_ret = -EINVAL;
                        goto unlock;
                }
                comp_ret = 0;
                dlen = PAGE_SIZE;
                dst = kmap_local_page(page);
        }

Looks very good to me. Much cleaner than the V2. Thanks for adopting that.

Quoted 11 lines
Quoted 10 lines

Another point I would like to make is that you currently make the cut off threshold as page size. The ideal threshold might be something slightly smaller than page size. The reason is that the zsmalloc has a fixed size chuck to store it. If your page is close to full, it will store the data in the same class as the full page. You are not gaining anything from zsmalloc if the store page compression ratio is 95%. That 5% will be in the waste in the zsmalloc class fragment anyway. So the trade off is, decompress 95% of a page vs memcpy 100% of a page. It is likely memcpy 100% is faster. I don't know the exact ideal cut off of threshold. If I had to guess, I would guess below 90%.

Quoted 1 line

Agreed, this could be another nice followup work to do.

Ack.

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

I can
be biased towards my own code :-).
I think we should treat the compression engine failure separately from the compression rate failure. The engine failure is rare and we should know about it as a real error. The compression rate failure is normal.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Again, I agree having the observability would be nice. I can add a new counter for that, like below attached patch, for example.

I would love that. Make that per memcg if possible :-)

As mentioned above, I would like to make only a global counter on debugfs for now, if you don't mind. Let me know if you mind.

I don't mind. We can take the incremental approach.

Chris

maintainer
#13 lore raw
In reply to Chris Li
Quoted 38 lines · Chris Li

Hi SeongJae,

Thanks for doing that. I am working on something related to this as well. Please consider CC me on your next version.

Quoted 15 lines · SeongJae Park

When zswap writeback is enabled and it fails compressing a given page, the page is swapped out to the backing swap device. This behavior breaks the zswap's writeback LRU order, and hence users can experience unexpected latency spikes. If the page is compressed without failure, but results in a size of PAGE_SIZE, the LRU order is kept, but the decompression overhead for loading the page back on the later access is unnecessary.

Keep the LRU order and optimize unnecessary decompression overheads in those cases, by storing the original content as-is in zpool. The length field of zswap_entry will be set appropriately, as PAGE_SIZE, Hence whether it is saved as-is or not (whether decompression is unnecessary) is identified by 'zswap_entry->length == PAGE_SIZE'.

Quoted 3 lines

Because the uncompressed data is saved in zpool, same to the compressed ones, this introduces no change in terms of memory management including movability and migratability of involved pages.

If you store uncompressed data in the zpool, zpool has metadata overhead, e.g. allocating the entry->handle for uncompressed pages. If the page is not compressed, another idea is just skip the zpool, store it as a page in the zswap entry as page. We can make a union of entry->handle and entry->incompressble_page. If entry->length == PAGE_SIZE, use entry->incompressable_page as a page.

The pros is that, on the page fault, there is no need to allocate a new page. You can just move the uncompressed_page into the swap_cache. You save one memcpy on the page fault as well. That will make the incompressible page fault behave very similar to the minor page fault.

Quoted 6 lines

The cons is that, now zpool does not account for uncompressed pages, on the second thought, that can be a con as well, the page is not compressed, why should it account as compressed pages?

I know Hugh has some idea to store incompressible pages in the swap cache as well. Hugh?

I've also proposed that approach internally - keeping the page in swapcache, while adding them to the zswap LRU for writeback to disk (and so that we do not consider them for zswap again in the future).

But after a while, we decided against it, mostly due to the complexity of the solution. On the zswap side, we need to distinguish between the ordinary struct zswap_entry and the struct page on zswap's LRU list. Externally, we need to handle moving a page currently in the zswap LRU to the main memory anon LRUs too.

Migration is another concern. Zswap needs to be notified that the "backend" of a zswap entry has changed underneath it. Not impossible, but again that's just more surgery.

So we decided to start with a simple solution (this one), and iterate as issues cropped up. At least then, we have production justifications for any future improvements.

Quoted 201 lines
Quoted 9 lines

This change is also not increasing per zswap entry metadata overhead. But as the number of incompressible pages increases, total zswap metadata overhead is proportionally increased. The overhead should not be problematic in usual cases, since the zswap metadata for single zswap entry is much smaller than PAGE_SIZE, and in common zswap use cases there should be a sufficient amount of compressible pages. Also it can be mitigated by the zswap writeback.

Quoted 8 lines

When the writeback is disabled, the additional overhead could be problematic. For the case, keep the current behavior that just returns the failure and let swap_writeout() put the page back to the active LRU list in the case.

Knowing how many compression failures happened will be useful for future investigations. investigations. Add a new debugfs file, compress_fail, for the purpose.

Again, I think we should distinguish the crypto engine fail vs ration failure.

Quoted 161 lines

Tests
-----

I tested this patch using a simple self-written microbenchmark that is available at GitHub[1]. You can reproduce the test I did by executing run_tests.sh of the repo on your system. Note that the repo's documentation is not good as of this writing, so you may need to read and use the code.

The basic test scenario is simple. Run a test program making artificial accesses to memory having artificial content under memory.high-set memory limit and measure how many accesses were made in given time.

The test program repeatedly and randomly access three anonymous memory regions. The regions are all 500 MiB size, and accessed in the same probability. Two of those are filled up with a simple content that can easily be compressed, while the remaining one is filled up with a content that read from /dev/urandom, which is easy to fail at compressing to <PAGE_SIZE size. The program runs for two minutes and prints out the number of accesses made every five seconds.

The test script runs the program under below seven configurations.

- 0: memory.high is set to 2 GiB, zswap is disabled.
- 1-1: memory.high is set to 1350 MiB, zswap is disabled.
- 1-2: On 1-1, zswap is enabled without this patch.
- 1-3: On 1-2, this patch is applied.

For all zswap enabled cases, zswap shrinker is enabled.

Configuration '0' is for showing the original memory performance. Configurations 1-1, 1-2 and 1-3 are for showing the performance of swap, zswap, and this patch under a level of memory pressure (~10% of working set).

Because the per-5 seconds performance is not very reliable, I measured the average of that for the last one minute period of the test program run. I also measured a few vmstat counters including zswpin, zswpout, zswpwb, pswpin and pswpout during the test runs.

The measurement results are as below. To save space, I show performance numbers that are normalized to that of the configuration '0' (no memory pressure), only. The averaged accesses per 5 seconds of configuration '0' was 36493417.75.

    config            0       1-1     1-2      1-3
    perf_normalized   1.0000  0.0057  0.0235   0.0367
    perf_stdev_ratio  0.0582  0.0652  0.0167   0.0346
    zswpin            0       0       3548424  1999335
    zswpout           0       0       3588817  2361689
    zswpwb            0       0       10214    340270
    pswpin            0       485806  772038   340967
    pswpout           0       649543  144773   340270

'perf_normalized' is the performance metric, normalized to that of configuration '0' (no pressure). 'perf_stdev_ratio' is the standard deviation of the averaged data points, as a ratio to the averaged metric

value.  For example, configuration '0' performance was showing 5.8%
stdev.  Configurations 1-1 and 1-3 were having about 6.5% and 6.1%
stdev.  Also the results were highly variable between multiple runs.  So

this result is not very stable but just showing ball park figures. Please keep this in your mind when reading these results.

Under about 10% of working set memory pressure, the performance was dropped to about 0.57% of no-pressure one, when the normal swap is used (1-1). Note that ~10% working set pressure is already extreme, at least on this test setup. No one would desire system setups that can degrade performance to 0.57% of the best case.

By turning zswap on (1-2), the performance was improved about 4x, resulting in about 2.35% of no-pressure one. Because of the incompressible pages in the third memory region, a significant amount of (non-zswap) swap I/O operations were made, though.

By applying this patch (1-3), about 56% performance improvement was made, resulting in about 3.67% of no-pressure one. Reduced pswpin of 1-3 compared to 1-2 let us see where this improvement came from.

Related Works
-------------

This is not an entirely new attempt. Nhat Pham and Takero Funaki tried very similar approaches in October 2023[2] and April 2024[3], respectively. The two approaches didn't get merged mainly due to the metadata overhead concern. I described why I think that shouldn't be a problem for this change, which is automatically disabled when writeback is disabled, at the beginning of this changelog.

This patch is not particularly different from those, and actually built upon those. I wrote this from scratch again, though. Hence adding Suggested-by tags for them. Actually Nhat first suggested this to me offlist.

Historically, writeback disabling was introduced partially as a way to solve the LRU order issue. Yosry pointed out[4] this is still suboptimal when the incompressible pages are cold, since the incompressible pages will continuously be tried to be zswapped out, and burn CPU cycles for compression attempts that will anyway fail. One imaginable solution for the problem is reusing the swapped-out page and its struct page to store in the zswap pool. But that's out of the scope of this patch.

[1] https://github.com/sjp38/eval_zswap/blob/master/run.sh
[2] https://lore.kernel.org/[email protected]
[3] https://lore.kernel.org/[email protected]
[4] https://lore.kernel.org/CAJD7tkZXS-UJVAFfvxJ0nNgTzWBiqepPYA4hEozi01_qktkitg@mail.gmail.com

---
Changes from v1
(https://lore.kernel.org/[email protected])
- Optimize out memcpy() per incompressible page saving, using k[un]map_local().
- Add a debugfs file for counting compression failures.
- Use a clear form of a ternary operation.
- Add the history of writeback disabling with a link.
- Wordsmith comments.

Changes from RFC v2
(https://lore.kernel.org/[email protected])
- Fix race conditions at decompressed pages identification.
- Remove the parameter and make saving as-is the default behavior.
- Open-code main changes.
- Clarify there is no memory management changes on the cover letter.
- Remove 20% pressure case from test results, since it is arguably too extreme and only adds confusion.
- Drop RFC tag.

Changes from RFC v1
(https://lore.kernel.org/[email protected])
- Consider PAGE_SIZE compression successes as failures.
- Use zpool for storing incompressible pages.
- Test with zswap shrinker enabled.
- Wordsmith changelog and comments.
- Add documentation of save_incompressible_pages parameter.

 mm/zswap.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)
mm/zswap.c +2 -2
index 3c0fd8a13718..0fb940d03268 100644
@@ -60,6 +60,8 @@ static u64 zswap_written_back_pages;
 static u64 zswap_reject_reclaim_fail;
 /* Store failed due to compression algorithm failure */
 static u64 zswap_reject_compress_fail;
+/* Compression into a size of <PAGE_SIZE failed */
+static u64 zswap_compress_fail;
 /* Compressed page was too big for the allocator to (optimally) store */
 static u64 zswap_reject_compress_poor;
 /* Load or writeback failed due to decompression failure */
@@ -976,8 +978,26 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry,
         */
        comp_ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait);
        dlen = acomp_ctx->req->dlen;
-       if (comp_ret)
-               goto unlock;

mm/zswap.c static bool zswap_compress(struct page *page, struct zswap_entry *entry, · line 990

Quoted 10 lines
mm/zswap.c +10 -0
+
+       /*
+        * If a page cannot be compressed into a size smaller than PAGE_SIZE,
+        * save the content as is without a compression, to keep the LRU order
+        * of writebacks.  If writeback is disabled, reject the page since it
+        * only adds metadata overhead.  swap_writeout() will put the page back
+        * to the active LRU list in the case.
+        */
+       if (comp_ret || dlen >= PAGE_SIZE) {
+               zswap_compress_fail++;

I feel that mixing comp_ret and dlen size if tested here complicates the nested if statement and the behavior as well.
One behavior change is that, if the comp_ret != 0, it means the compression crypt engine has some error. e.g. have some bad chip always fail the compression. That is a different error case than the compression was successful and the compression rate is not good. In this case the patch makes the page store in zpool for cryto engine failure, which does not help.

Quoted 2 lines

Is there any reason you want to store the page in zpool when the compression engine (not the ratio) fails?

It helps if you want to write them back to disk later, in the LRU order. SJ pointed that out in the patch changelog, I believe.

#14 lore raw
Quoted 9 lines · Nhat Pham
Quoted 2 lines

I know Hugh has some idea to store incompressible pages in the swap cache as well. Hugh?

I've also proposed that approach internally - keeping the page in swapcache, while adding them to the zswap LRU for writeback to disk (and so that we do not consider them for zswap again in the future).

But after a while, we decided against it, mostly due to the complexity of the solution. On the zswap side, we need to distinguish between the

Google actually has an internal patch to keep incompressible pages in separate LRU out of zswap. But that breaks the zswap LRU order as well. If there is interest and I can find the time, I can send it out for note comparison purposes. I do see the value of maintaining the LRU in the zswap tier as a whole.

Quoted 7 lines

ordinary struct zswap_entry and the struct page on zswap's LRU list. Externally, we need to handle moving a page currently in the zswap LRU to the main memory anon LRUs too.

Migration is another concern. Zswap needs to be notified that the "backend" of a zswap entry has changed underneath it. Not impossible, but again that's just more surgery.

Ack. We might need to get that operation inside zsmalloc.

Quoted 4 lines

So we decided to start with a simple solution (this one), and iterate as issues cropped up. At least then, we have production justifications for any future improvements.

Ack.

Chris

maintainer
#15 lore raw
Quoted 9 lines · Chris Li
Quoted 7 lines · Nhat Pham
Quoted 2 lines

I know Hugh has some idea to store incompressible pages in the swap cache as well. Hugh?

I've also proposed that approach internally - keeping the page in swapcache, while adding them to the zswap LRU for writeback to disk (and so that we do not consider them for zswap again in the future).

Quoted 8 lines
Quoted 2 lines

But after a while, we decided against it, mostly due to the complexity of the solution. On the zswap side, we need to distinguish between the

Google actually has an internal patch to keep incompressible pages in separate LRU out of zswap. But that breaks the zswap LRU order as well. If there is interest and I can find the time, I can send it out for note comparison purposes. I do see the value of maintaining the LRU in the zswap tier as a whole.

It would be very valuable. Much appreciated, Chris!

And yes, we also discussed that approach too. What I described above was an attempt to get the best-of-all-world.

There's a couple of desirata:

1. Maintain LRU ordering, in case we want to do zswap writeback.
2. Do not retry incompressible pages, until they are accessed.
3. Minimize the cost of page fault (memcpy, page allocation/free), as much as possible.
4. Do not retry incompressible pages, until they are dirtied (a stronger guarantee than 2).
5. Keep the stored data migratable.

In the end, it got too complicated. So we decided to go for 1, 2, and 5, with this approach. Regarding 3, this is still an improvement over vanilla zswap (which writes to disk, and tends to have an even higher cost of page fault).

I believe Google's approach gets us 2, 3, 4.

Quoted 19 lines
Quoted 8 lines

ordinary struct zswap_entry and the struct page on zswap's LRU list. Externally, we need to handle moving a page currently in the zswap LRU to the main memory anon LRUs too.

Migration is another concern. Zswap needs to be notified that the "backend" of a zswap entry has changed underneath it. Not impossible, but again that's just more surgery.

Ack. We might need to get that operation inside zsmalloc.

Quoted 4 lines

So we decided to start with a simple solution (this one), and iterate as issues cropped up. At least then, we have production justifications for any future improvements.

Ack.

Chris

#16 lore raw
In reply to Chris Li
Quoted 3 lines · Chris Li

I know Hugh has some idea to store incompressible pages in the swap cache as well. Hugh?

No, I don't have any idea or plan to store incompressible pages in the swap cache myself. But yes, incompressible pages could well be "stored" in the swap cache.

Hugh

#17 lore raw
Quoted 9 lines · Hugh Dickins
Quoted 3 lines · Chris Li

I know Hugh has some idea to store incompressible pages in the swap cache as well. Hugh?

No, I don't have any idea or plan to store incompressible pages in the swap cache myself. But yes, incompressible pages could well be "stored" in the swap cache.

Sorry Hugh, I don't mean to assign you more work. Just brainstorming. Yes it could. The pros will be no swap fault. The cons is breaking the zswap LRU ordering.

Chris

#18 lore raw
In reply to Chris Li
Quoted 7 lines · Chris Li

If you store uncompressed data in the zpool, zpool has metadata overhead, e.g. allocating the entry->handle for uncompressed pages. If the page is not compressed, another idea is just skip the zpool, store it as a page in the zswap entry as page. We can make a union of entry->handle and entry->incompressble_page. If entry->length == PAGE_SIZE, use entry->incompressable_page as a page.

The main problem being solved here is to avoid the scenario where the incompressible pages are being rotated in LRUs and zswapped multiple times and wasting CPU on compressing incompressible pages. SJ's approach solves the issue but with some memory overhead (zswap entry). With your suggestion and to solve the mentioned issue, we will need to change some core parts of reclaim (__remove_mapping()), LRU handling (swap cache pages not in LRUs) and refault (putting such pages back in LRU and should it handle read and write faults differently). So, the cons of that approach is more complex code.

Personally I would prefer a simple solution with some overhead over a more complicated and error prone solution without overhead. Or maybe you have a more simplied approach instead?

maintainer
#19 lore raw
Quoted 9 lines · Shakeel Butt
Quoted 7 lines · Chris Li

If you store uncompressed data in the zpool, zpool has metadata overhead, e.g. allocating the entry->handle for uncompressed pages. If the page is not compressed, another idea is just skip the zpool, store it as a page in the zswap entry as page. We can make a union of entry->handle and entry->incompressble_page. If entry->length == PAGE_SIZE, use entry->incompressable_page as a page.

Quoted 9 lines

The main problem being solved here is to avoid the scenario where the incompressible pages are being rotated in LRUs and zswapped multiple times and wasting CPU on compressing incompressible pages. SJ's approach solves the issue but with some memory overhead (zswap entry). With your suggestion and to solve the mentioned issue, we will need to change some core parts of reclaim (__remove_mapping()), LRU handling (swap cache pages not in LRUs) and refault (putting such pages back in LRU and should it handle read and write faults differently). So, the cons of that approach is more complex code.

What Chris is proposing would also fix that, even for configurations without writeback. So I'm not opposed to it.

However, for deployments where writeback *is* enabled, this code is an improvement over the status quo. And it's not in conflict with a broader fix for !writeback setups, so it's not an either-or scenario.

Specifically for the writeback case, the metadata overhead is not much of a concern: we can just write back more zswap tail to make up for it; the more important thing is that we can now do so in LRU order.

The premise being that writing an additional cold page from zswap to disk to make room for a slightly inefficiently stored warm page is better than rejecting and sending the *warm* page to disk instead.

So I agree with you Chris. But also think that's follow-up work for somebody who cares about the !writeback case.

#20 lore raw
Quoted 32 lines · Johannes Weiner
Quoted 9 lines · Shakeel Butt
Quoted 7 lines · Chris Li

If you store uncompressed data in the zpool, zpool has metadata overhead, e.g. allocating the entry->handle for uncompressed pages. If the page is not compressed, another idea is just skip the zpool, store it as a page in the zswap entry as page. We can make a union of entry->handle and entry->incompressble_page. If entry->length == PAGE_SIZE, use entry->incompressable_page as a page.

Quoted 9 lines

The main problem being solved here is to avoid the scenario where the incompressible pages are being rotated in LRUs and zswapped multiple times and wasting CPU on compressing incompressible pages. SJ's approach solves the issue but with some memory overhead (zswap entry). With your suggestion and to solve the mentioned issue, we will need to change some core parts of reclaim (__remove_mapping()), LRU handling (swap cache pages not in LRUs) and refault (putting such pages back in LRU and should it handle read and write faults differently). So, the cons of that approach is more complex code.

What Chris is proposing would also fix that, even for configurations without writeback. So I'm not opposed to it.

However, for deployments where writeback *is* enabled, this code is an improvement over the status quo. And it's not in conflict with a broader fix for !writeback setups, so it's not an either-or scenario.

Specifically for the writeback case, the metadata overhead is not much of a concern: we can just write back more zswap tail to make up for it; the more important thing is that we can now do so in LRU order.

Quoted 6 lines

The premise being that writing an additional cold page from zswap to disk to make room for a slightly inefficiently stored warm page is better than rejecting and sending the *warm* page to disk instead.

So I agree with you Chris. But also think that's follow-up work for somebody who cares about the !writeback case.

Thank you and I agree with your assessment.

Chris

#21 lore raw
In reply to Shakeel Butt
Quoted 10 lines · Shakeel Butt
Quoted 7 lines · Chris Li

If you store uncompressed data in the zpool, zpool has metadata overhead, e.g. allocating the entry->handle for uncompressed pages. If the page is not compressed, another idea is just skip the zpool, store it as a page in the zswap entry as page. We can make a union of entry->handle and entry->incompressble_page. If entry->length == PAGE_SIZE, use entry->incompressable_page as a page.

Quoted 7 lines

The main problem being solved here is to avoid the scenario where the incompressible pages are being rotated in LRUs and zswapped multiple times and wasting CPU on compressing incompressible pages. SJ's approach solves the issue but with some memory overhead (zswap entry). With your suggestion and to solve the mentioned issue, we will need to change some core parts of reclaim (__remove_mapping()), LRU handling (swap cache pages not in LRUs) and refault (putting such pages back in LRU and

No, that is not what I have in mind. I mean store a separate page and memcpy into it, keep the swap cache folio still reclaimed as normal. Not much different from a store in the zpool.

The cons really are migration etc. We might have to get it in zsmalloc later.

Quoted 6 lines

should it handle read and write faults differently). So, the cons of that approach is more complex code.

Personally I would prefer a simple solution with some overhead over a more complicated and error prone solution without overhead. Or maybe you have a more simplied approach instead?

Ack, there is more complexity to get the last bit of performance gain. Maybe in zsmalloc later.

Chris

maintainer
#22 lore raw
In reply to SeongJae Park
Quoted 56 lines · SeongJae Park

When zswap writeback is enabled and it fails compressing a given page, the page is swapped out to the backing swap device. This behavior breaks the zswap's writeback LRU order, and hence users can experience unexpected latency spikes. If the page is compressed without failure, but results in a size of PAGE_SIZE, the LRU order is kept, but the decompression overhead for loading the page back on the later access is unnecessary.

Keep the LRU order and optimize unnecessary decompression overheads in those cases, by storing the original content as-is in zpool. The length field of zswap_entry will be set appropriately, as PAGE_SIZE, Hence whether it is saved as-is or not (whether decompression is unnecessary) is identified by 'zswap_entry->length == PAGE_SIZE'.

Because the uncompressed data is saved in zpool, same to the compressed ones, this introduces no change in terms of memory management including movability and migratability of involved pages.

This change is also not increasing per zswap entry metadata overhead. But as the number of incompressible pages increases, total zswap metadata overhead is proportionally increased. The overhead should not be problematic in usual cases, since the zswap metadata for single zswap entry is much smaller than PAGE_SIZE, and in common zswap use cases there should be a sufficient amount of compressible pages. Also it can be mitigated by the zswap writeback.

When the writeback is disabled, the additional overhead could be problematic. For the case, keep the current behavior that just returns the failure and let swap_writeout() put the page back to the active LRU list in the case.

Knowing how many compression failures happened will be useful for future investigations. investigations. Add a new debugfs file, compress_fail, for the purpose.

Tests
-----

I tested this patch using a simple self-written microbenchmark that is available at GitHub[1]. You can reproduce the test I did by executing run_tests.sh of the repo on your system. Note that the repo's documentation is not good as of this writing, so you may need to read and use the code.

The basic test scenario is simple. Run a test program making artificial accesses to memory having artificial content under memory.high-set memory limit and measure how many accesses were made in given time.

The test program repeatedly and randomly access three anonymous memory regions. The regions are all 500 MiB size, and accessed in the same probability. Two of those are filled up with a simple content that can easily be compressed, while the remaining one is filled up with a content that read from /dev/urandom, which is easy to fail at compressing to <PAGE_SIZE size. The program runs for two minutes and prints out the number of accesses made every five seconds.

Quoted 1 line

The test script runs the program under below seven configurations.

four?

Quoted 4 lines

- 0: memory.high is set to 2 GiB, zswap is disabled.
- 1-1: memory.high is set to 1350 MiB, zswap is disabled.
- 1-2: On 1-1, zswap is enabled without this patch.
- 1-3: On 1-2, this patch is applied.

I'm not sure 0 and 1-1 are super interesting, since we care about zswap performance under pressure before and after the patch.

Quoted 1 line

For all zswap enabled cases, zswap shrinker is enabled.

It would, however, be good to test without the shrinker as well, since it's currently optional and not enabled by default.

The performance with the shrinker looks great, but if it regresses for setups without the shrinker we need additional gating.

#23 lore raw
Quoted 14 lines · On Thu, 14 Aug 2025 12:23:50 -0400 Johannes Weiner <[email protected]> wrote:
Quoted 13 lines · SeongJae Park

When zswap writeback is enabled and it fails compressing a given page, the page is swapped out to the backing swap device. This behavior breaks the zswap's writeback LRU order, and hence users can experience unexpected latency spikes. If the page is compressed without failure, but results in a size of PAGE_SIZE, the LRU order is kept, but the decompression overhead for loading the page back on the later access is unnecessary.

Keep the LRU order and optimize unnecessary decompression overheads in those cases, by storing the original content as-is in zpool. The length field of zswap_entry will be set appropriately, as PAGE_SIZE, Hence whether it is saved as-is or not (whether decompression is unnecessary) is identified by 'zswap_entry->length == PAGE_SIZE'.

[...]

Quoted 21 lines
Quoted 21 lines

Tests
-----

I tested this patch using a simple self-written microbenchmark that is available at GitHub[1]. You can reproduce the test I did by executing run_tests.sh of the repo on your system. Note that the repo's documentation is not good as of this writing, so you may need to read and use the code.

The basic test scenario is simple. Run a test program making artificial accesses to memory having artificial content under memory.high-set memory limit and measure how many accesses were made in given time.

The test program repeatedly and randomly access three anonymous memory regions. The regions are all 500 MiB size, and accessed in the same probability. Two of those are filled up with a simple content that can easily be compressed, while the remaining one is filled up with a content that read from /dev/urandom, which is easy to fail at compressing to <PAGE_SIZE size. The program runs for two minutes and prints out the number of accesses made every five seconds.

Quoted 3 lines
Quoted 1 line

The test script runs the program under below seven configurations.

four?

Nice catch, thank you! I will fix this in the next version.

Quoted 8 lines
Quoted 5 lines

- 0: memory.high is set to 2 GiB, zswap is disabled.
- 1-1: memory.high is set to 1350 MiB, zswap is disabled.
- 1-2: On 1-1, zswap is enabled without this patch.
- 1-3: On 1-2, this patch is applied.

I'm not sure 0 and 1-1 are super interesting, since we care about zswap performance under pressure before and after the patch.

You're right, the main focus should be 1-2 vs 1-3.

I added 0 and 1-1, too, though, just for showing how artificial and micro the given test setup is.

That is, as I also mentioned on the original mail, this test setup is for pretty extreme pressure case. To quote,

        config            0       1-1     1-2      1-3
        perf_normalized   1.0000  0.0057  0.0235   0.0367

[...]
Note that ~10% working set pressure is already extreme, at least on this test setup. No one would desire system setups that can degrade performance to 0.57% of the best case.

Enabling zswap improves the performance to 2.35% of the ideal setup, and this patch further improves it to 3.67%. But still no one would want a setup that achieves only 3.67% of the ideal performance. The performance itself is also a pure memory random access performance, which is again far from those for real workloads, though.

So, this is a microbenchmarking test that pretty far from the real world.

I will add more description about the intention of the configs in the next version. Let me know if you think it is only disturbing reading of the patch and better to be just dropped.

Quoted 5 lines
Quoted 2 lines

For all zswap enabled cases, zswap shrinker is enabled.

It would, however, be good to test without the shrinker as well, since it's currently optional and not enabled by default.

Agreed. Actually we ran without shrinker for RFC v1[1] of this patch. The version also showed performance improvement in the configuration. Since we made no big change after the version, I expect no big difference.

Nonetheless, I will run the test with zswap shrinker disabled setup again with this version, and share the results. Unless the results is far from my expectation, I will directly post the next version of this patch with abovely promised changelog updates.

Quoted 3 lines

The performance with the shrinker looks great, but if it regresses for setups without the shrinker we need additional gating.

Agreed. If I find a regression from the abovely promised rerun of the test, I will share that as a reply to this thread, and start a discussion for the possible additional gating.

[1] https://lore.kernel.org/[email protected]

Thanks,
SJ