Skip to contentSkip to conversation

page_io: zswap: do not crash the kernel on decompression failure

PATCH Applied Nhat Pham 5 participants 9 messages Mar 6, 2025 – May 12, 2025

Subsystems (3)
lore
Share & tools
lore

Nhat Pham

author maintainer
#0 lore raw

PATCH v4 1/1 page_io: zswap: do not crash the kernel on decompression failure

Currently, we crash the kernel when a decompression failure occurs in zswap (either because of memory corruption, or a bug in the compression algorithm). This is overkill. We should only SIGBUS the unfortunate process asking for the zswap entry on zswap load, and skip the corrupted entry in zswap writeback.

See [1] for a recent upstream discussion about this.

The zswap writeback case is relatively straightforward to fix. For the zswap_load() case, we change the return behavior:

* Return 0 on success.
* Return -ENOENT (with the folio locked) if zswap does not own the swapped out content.
* Return -EIO if zswap owns the swapped out content, but encounters a decompression failure for some reasons. The folio will be unlocked, but not be marked up-to-date, which will eventually cause the process requesting the page to SIGBUS (see the handling of not-up-to-date folio in do_swap_page() in mm/memory.c), without crashing the kernel.
* Return -EINVAL if we encounter a large folio, as large folio should not be swapped in while zswap is being used. Similar to the -EIO case, we also unlock the folio but do not mark it as up-to-date to SIGBUS the faulting process.

As a side effect, we require one extra zswap tree traversal in the load and writeback paths. Quick benchmarking on a kernel build test shows no performance difference:

With the new scheme:
real: mean: 125.1s, stdev: 0.12s
user: mean: 3265.23s, stdev: 9.62s
sys: mean: 2156.41s, stdev: 13.98s

The old scheme:
real: mean: 125.78s, stdev: 0.45s
user: mean: 3287.18s, stdev: 5.95s
sys: mean: 2177.08s, stdev: 26.52s

[1]: https://lore.kernel.org/all/[email protected]/

---
 include/linux/zswap.h |   6 +--
 mm/page_io.c          |   6 +--
 mm/zswap.c            | 118 +++++++++++++++++++++++++++++-------------
 3 files changed, 87 insertions(+), 43 deletions(-)
include/linux/zswap.h +3 -3
index d961ead91bf1..30c193a1207e 100644
@@ -26,7 +26,7 @@ struct zswap_lruvec_state {
 
 unsigned long zswap_total_pages(void);
 bool zswap_store(struct folio *folio);
-bool zswap_load(struct folio *folio);
+int zswap_load(struct folio *folio);
 void zswap_invalidate(swp_entry_t swp);
 int zswap_swapon(int type, unsigned long nr_pages);
 void zswap_swapoff(int type);
@@ -44,9 +44,9 @@ static inline bool zswap_store(struct folio *folio)
 	return false;
 }
 
-static inline bool zswap_load(struct folio *folio)
+static inline int zswap_load(struct folio *folio)
 {
-	return false;
+	return -ENOENT;
 }
 
 static inline void zswap_invalidate(swp_entry_t swp) {}
mm/page_io.c +3 -3
index 9b983de351f9..4bce19df557b 100644
@@ -638,11 +638,11 @@ void swap_read_folio(struct folio *folio, struct swap_iocb **plug)
 	if (swap_read_folio_zeromap(folio)) {
 		folio_unlock(folio);
 		goto finish;
-	} else if (zswap_load(folio)) {
-		folio_unlock(folio);
-		goto finish;
 	}
 
+	if (zswap_load(folio) != -ENOENT)
+		goto finish;
+
 	/* We have to read from slower devices. Increase zswap protection. */
 	zswap_folio_swapin(folio);
 
mm/zswap.c +81 -37

204 lines not on this page · raw message

base-commit: 36768063c930199d5e01cde4d7a14fae3ccad51c

Signature · 1 line
2.43.5
Acked-by on v4 1/1
#1 lore raw
Quoted 38 lines · Nhat Pham

Currently, we crash the kernel when a decompression failure occurs in zswap (either because of memory corruption, or a bug in the compression algorithm). This is overkill. We should only SIGBUS the unfortunate process asking for the zswap entry on zswap load, and skip the corrupted entry in zswap writeback.

See [1] for a recent upstream discussion about this.

The zswap writeback case is relatively straightforward to fix. For the zswap_load() case, we change the return behavior:

* Return 0 on success.
* Return -ENOENT (with the folio locked) if zswap does not own the swapped out content.
* Return -EIO if zswap owns the swapped out content, but encounters a decompression failure for some reasons. The folio will be unlocked, but not be marked up-to-date, which will eventually cause the process requesting the page to SIGBUS (see the handling of not-up-to-date folio in do_swap_page() in mm/memory.c), without crashing the kernel.
* Return -EINVAL if we encounter a large folio, as large folio should not be swapped in while zswap is being used. Similar to the -EIO case, we also unlock the folio but do not mark it as up-to-date to SIGBUS the faulting process.

As a side effect, we require one extra zswap tree traversal in the load and writeback paths. Quick benchmarking on a kernel build test shows no performance difference:

With the new scheme:
real: mean: 125.1s, stdev: 0.12s
user: mean: 3265.23s, stdev: 9.62s
sys: mean: 2156.41s, stdev: 13.98s

The old scheme:
real: mean: 125.78s, stdev: 0.45s
user: mean: 3287.18s, stdev: 5.95s
sys: mean: 2177.08s, stdev: 26.52s

Quoted 6 lines

[1]: https://lore.kernel.org/all/[email protected]/

Couple of nits below, but otherwise LGTM:

(I did expect the swap zeromap change in the same series, so if you send it separately make sure to mention it's on top of this one because they will conflict otherwise)

[..]

mm/zswap.c bool zswap_store(struct folio *folio) · line 1632

Quoted 7 lines
mm/zswap.c +2 -1
@@ -1606,7 +1628,26 @@ bool zswap_store(struct folio *folio)
 	return ret;
 }
 
-bool zswap_load(struct folio *folio)
+/**
+ * zswap_load() - load a page from zswap

nit: folio

mm/zswap.c bool zswap_store(struct folio *folio) · line 1635

Quoted 3 lines
mm/zswap.c +3 -0
+ * @folio: folio to load
+ *
+ * Return: 0 on success, or one of the following error codes:

nit: Maybe worth mentioning that the folio is unlocked and marked uptodate on success for completeness.

Quoted 15 lines
mm/zswap.c +15 -0
+ *
+ *  -EIO: if the swapped out content was in zswap, but could not be loaded
+ *  into the page due to a decompression failure. The folio is unlocked, but
+ *  NOT marked up-to-date, so that an IO error is emitted (e.g. do_swap_page()
+ *  will SIGBUS).
+ *
+ *  -EINVAL: if the swapped out content was in zswap, but the page belongs
+ *  to a large folio, which is not supported by zswap. The folio is unlocked,
+ *  but NOT marked up-to-date, so that an IO error is emitted (e.g.
+ *  do_swap_page() will SIGBUS).
+ *
+ *  -ENOENT: if the swapped out content was not in zswap. The folio remains
+ *  locked on return.
+ */
+int zswap_load(struct folio *folio)

[..]

author maintainer
#2 lore raw
Quoted 10 lines · Yosry Ahmed

On Thu, Mar 06, 2025 at 12:50:10PM -0800, Nhat Pham wrote:

Couple of nits below, but otherwise LGTM:

(I did expect the swap zeromap change in the same series, so if you send it separately make sure to mention it's on top of this one because they will conflict otherwise)

Yeah of course. I actually have already finished that second patch, but wanna spend a bit more time proofreading things :)

Quoted 12 lines

[..]

mm/zswap.c bool zswap_store(struct folio *folio) · line 1632

Quoted 7 lines
mm/zswap.c +2 -1
@@ -1606,7 +1628,26 @@ bool zswap_store(struct folio *folio)
      return ret;
 }
 
-bool zswap_load(struct folio *folio)
+/**
+ * zswap_load() - load a page from zswap

nit: folio

Quoted 6 lines

mm/zswap.c bool zswap_store(struct folio *folio) · line 1635

Quoted 3 lines
mm/zswap.c +3 -0
+ * @folio: folio to load
+ *
+ * Return: 0 on success, or one of the following error codes:

nit: Maybe worth mentioning that the folio is unlocked and marked uptodate on success for completeness.

Will do!

Quoted 17 lines
Quoted 16 lines
mm/zswap.c +15 -0
+ *
+ *  -EIO: if the swapped out content was in zswap, but could not be loaded
+ *  into the page due to a decompression failure. The folio is unlocked, but
+ *  NOT marked up-to-date, so that an IO error is emitted (e.g. do_swap_page()
+ *  will SIGBUS).
+ *
+ *  -EINVAL: if the swapped out content was in zswap, but the page belongs
+ *  to a large folio, which is not supported by zswap. The folio is unlocked,
+ *  but NOT marked up-to-date, so that an IO error is emitted (e.g.
+ *  do_swap_page() will SIGBUS).
+ *
+ *  -ENOENT: if the swapped out content was not in zswap. The folio remains
+ *  locked on return.
+ */
+int zswap_load(struct folio *folio)

[..]

author maintainer
#3 lore raw

PATCH v4 1/1 page_io: zswap: do not crash the kernel on decompression failure (fix)

In reply to Nhat Pham

Fix the documentation of zswap_load() - should say load a folio (rather than a page), and should be explicit that the folio is unlocked and marked up-to-date in the success case.

---
 mm/zswap.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
mm/zswap.c +3 -2
index 799b22c19b5e..c470073c17cc 100644
@@ -1629,10 +1629,11 @@ bool zswap_store(struct folio *folio)
 }
 
 /**
- * zswap_load() - load a page from zswap
+ * zswap_load() - load a folio from zswap
  * @folio: folio to load
  *
- * Return: 0 on success, or one of the following error codes:
+ * Return: 0 on success, with the folio unlocked and marked up-to-date, or one
+ * of the following error codes:
  *
  *  -EIO: if the swapped out content was in zswap, but could not be loaded
  *  into the page due to a decompression failure. The folio is unlocked, but
Signature · 1 line
2.43.5
maintainer Acked-by on v4 1/1
#4 lore raw
In reply to Nhat Pham
Quoted 38 lines · Nhat Pham

Currently, we crash the kernel when a decompression failure occurs in zswap (either because of memory corruption, or a bug in the compression algorithm). This is overkill. We should only SIGBUS the unfortunate process asking for the zswap entry on zswap load, and skip the corrupted entry in zswap writeback.

See [1] for a recent upstream discussion about this.

The zswap writeback case is relatively straightforward to fix. For the zswap_load() case, we change the return behavior:

* Return 0 on success.
* Return -ENOENT (with the folio locked) if zswap does not own the swapped out content.
* Return -EIO if zswap owns the swapped out content, but encounters a decompression failure for some reasons. The folio will be unlocked, but not be marked up-to-date, which will eventually cause the process requesting the page to SIGBUS (see the handling of not-up-to-date folio in do_swap_page() in mm/memory.c), without crashing the kernel.
* Return -EINVAL if we encounter a large folio, as large folio should not be swapped in while zswap is being used. Similar to the -EIO case, we also unlock the folio but do not mark it as up-to-date to SIGBUS the faulting process.

As a side effect, we require one extra zswap tree traversal in the load and writeback paths. Quick benchmarking on a kernel build test shows no performance difference:

With the new scheme:
real: mean: 125.1s, stdev: 0.12s
user: mean: 3265.23s, stdev: 9.62s
sys: mean: 2156.41s, stdev: 13.98s

The old scheme:
real: mean: 125.78s, stdev: 0.45s
user: mean: 3287.18s, stdev: 5.95s
sys: mean: 2177.08s, stdev: 26.52s

Quoted 6 lines

[1]: https://lore.kernel.org/all/[email protected]/

reviewer Reviewed-by on v4 1/1
#5 lore raw
In reply to Nhat Pham
Quoted 38 lines · Nhat Pham

Currently, we crash the kernel when a decompression failure occurs in zswap (either because of memory corruption, or a bug in the compression algorithm). This is overkill. We should only SIGBUS the unfortunate process asking for the zswap entry on zswap load, and skip the corrupted entry in zswap writeback.

See [1] for a recent upstream discussion about this.

The zswap writeback case is relatively straightforward to fix. For the zswap_load() case, we change the return behavior:

* Return 0 on success.
* Return -ENOENT (with the folio locked) if zswap does not own the swapped out content.
* Return -EIO if zswap owns the swapped out content, but encounters a decompression failure for some reasons. The folio will be unlocked, but not be marked up-to-date, which will eventually cause the process requesting the page to SIGBUS (see the handling of not-up-to-date folio in do_swap_page() in mm/memory.c), without crashing the kernel.
* Return -EINVAL if we encounter a large folio, as large folio should not be swapped in while zswap is being used. Similar to the -EIO case, we also unlock the folio but do not mark it as up-to-date to SIGBUS the faulting process.

As a side effect, we require one extra zswap tree traversal in the load and writeback paths. Quick benchmarking on a kernel build test shows no performance difference:

With the new scheme:
real: mean: 125.1s, stdev: 0.12s
user: mean: 3265.23s, stdev: 9.62s
sys: mean: 2156.41s, stdev: 13.98s

The old scheme:
real: mean: 125.78s, stdev: 0.45s
user: mean: 3287.18s, stdev: 5.95s
sys: mean: 2177.08s, stdev: 26.52s

Quoted 6 lines

[1]: https://lore.kernel.org/all/[email protected]/

Thanks!

#6 lore raw
In reply to Nhat Pham

mm/zswap.c static bool zswap_compress(struct page *page, struct zswap_entry *entry, · line 990

Quoted 2 lines · Nhat Pham
mm/zswap.c +1 -1
-static void zswap_decompress(struct zswap_entry *entry, struct folio *folio)
+static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio)

Hm, why do it this way? I had it as:

-static void zswap_decompress(struct zswap_entry *entry, struct folio *folio) +static int zswap_decompress(struct zswap_entry *entry, struct folio *folio)
...

+       err = crypto_acomp_decompress(acomp_ctx->req);
+       err = crypto_wait_req(err, &acomp_ctx->wait);
+       if (!err && acomp_ctx->req->dlen != PAGE_SIZE)

+ err = -EIO;

which allows us to return something more meaningful than -EIO. Or is doing that a bad idea and we should squash all decompression failures to EIO?

(also i really dislike the chained approach:

        decomp_ret = crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait);

that's much harder to understand than the two lines i have above)

author maintainer
#7 lore raw
Quoted 15 lines · Matthew Wilcox

mm/zswap.c static bool zswap_compress(struct page *page, struct zswap_entry *entry, · line 990

Quoted 2 lines · Nhat Pham
mm/zswap.c +1 -1
-static void zswap_decompress(struct zswap_entry *entry, struct folio *folio)
+static bool zswap_decompress(struct zswap_entry *entry, struct folio *folio)

Hm, why do it this way? I had it as:

(quoted diff) +5 -1
-static void zswap_decompress(struct zswap_entry *entry, struct folio *folio)
+static int zswap_decompress(struct zswap_entry *entry, struct folio *folio)
...
+       err = crypto_acomp_decompress(acomp_ctx->req);
+       err = crypto_wait_req(err, &acomp_ctx->wait);
+       if (!err && acomp_ctx->req->dlen != PAGE_SIZE)
+               err = -EIO;
Quoted 3 lines

which allows us to return something more meaningful than -EIO. Or is doing that a bad idea and we should squash all decompression failures to EIO?

Not a bad idea, just no usecase yet. From the POV of zswap_decompress()'s callers, any decompression failure is handled the same way, no matter the cause.

Where it might be useful is the debug print statement right below this. However, zstd and lz4 only return 0 or -EINVAL, I think - not super useful.

https://github.com/torvalds/linux/blob/master/crypto/zstd.c#L163-L165
https://github.com/torvalds/linux/blob/master/crypto/lz4.c#L61

Not sure about other compressors.

Quoted 6 lines

(also i really dislike the chained approach:

        decomp_ret = crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait);

that's much harder to understand than the two lines i have above)

I can send a style fix sometimes if people dislike this - no big deal :)

BTW, hopefully you don't mind the Suggested-by: tag. I know it deviated a bit from your original suggestion, but the main spirit of that suggestion remains, so I feel like I should credit you and Yosry.

#8 lore raw
Quoted 4 lines · Nhat Pham

Not a bad idea, just no usecase yet. From the POV of zswap_decompress()'s callers, any decompression failure is handled the same way, no matter the cause.

Quoted 8 lines

Where it might be useful is the debug print statement right below this. However, zstd and lz4 only return 0 or -EINVAL, I think - not super useful.

https://github.com/torvalds/linux/blob/master/crypto/zstd.c#L163-L165
https://github.com/torvalds/linux/blob/master/crypto/lz4.c#L61

Not sure about other compressors.

OK, fair enough. If there's no better information to return, then this way is fine.

Quoted 11 lines
Quoted 5 lines

(also i really dislike the chained approach:

        decomp_ret = crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait);

that's much harder to understand than the two lines i have above)

I can send a style fix sometimes if people dislike this - no big deal :)

BTW, hopefully you don't mind the Suggested-by: tag. I know it deviated a bit from your original suggestion, but the main spirit of that suggestion remains, so I feel like I should credit you and Yosry.

The Suggested-by is appropriate, I think.