lib/842: reject malformed streams before invalid memory accesses

sw842_decompress() backs the crypto 842 algorithm, the NX-842 software fallback and zram's 842 backend. Two malformed-stream checks are missing:

- indexed and short-data templates can write beyond the caller's stated output capacity; and
- a repeat after one to seven bytes of short data reads before the output buffer.

The first underflows the unsigned remaining length, so every later capacity check passes. The decoder keeps writing past the destination and can return success with an output length the caller will trust.

The decoder already returns errors for format, capacity and CRC problems; these two cases perform invalid accesses instead.

Patches 1 and 2 add the checks. After them every output write site is capacity-checked, repeat cannot read before the output, and all input reads stay bounded. Patch 3 adds KUnit cases for the three malformed streams and for three valid streams at the exact acceptance boundaries, so the new rejections cannot regress into off-by-one.

The KMSAN reports from
syzbot+e774233ff687aada969e and syzbot+8f77ff6144a73f0cf71b are unrelated. They trace poison from uninitialized swapped-page storage, not these bounds failures.

KUnit under QEMU 10.2.1 TCG, two vCPUs:

baseline fixed

  i386              3/6          6/6
  x86_64            3/6          6/6

Baseline runs applied patch 3 alone: the three boundary cases pass, the three malformed streams wrongly return success. A fixed x86_64 lockdep kernel also passes 6/6.

Separately, an x86_64 KASAN QEMU run exercised all three malformed cases through zram's compressed writeback path by corrupting the backing-disk contents after writeback. The baseline reported two out-of-bounds writes and one out-of-bounds read; fixed readback returned -EIO in all three cases without a KASAN report.

Karl Mehltretter (3):
lib/842: reject output overflows from index and short data lib/842: require a complete history block for repeat templates lib/842: add KUnit tests for the decompressor

 MAINTAINERS                      |   1 +
 lib/842/842_decompress.c         |   7 +-
 lib/Kconfig.debug                |  15 ++++
 lib/tests/842_decompress_kunit.c | 144 +++++++++++++++++++++++++++++++
 lib/tests/Makefile               |   1 +
 5 files changed, 167 insertions(+), 1 deletion(-)
 create mode 100644 lib/tests/842_decompress_kunit.c

base-commit: cf72cbb39da84b6f02f90c07f33b102fc10b16f0

2.53.0

The output length passed to sw842_decompress() is the caller's buffer capacity. Indexed copies write 2, 4 or 8 bytes and short-data templates write up to 7, but neither checks that capacity before writing and decrementing p->olen.

Overwriting an undersized destination then underflows p->olen, which is unsigned, so every later bounds check in the stream passes. Subsequent operations keep writing past the destination, and a matching CRC lets sw842_decompress() return success with an output length larger than the capacity the caller supplied.

This is reachable through zram's compressed writeback path. With 842 selected, targeted corruption of the compressed data on its backing device made a KASAN kernel report vmalloc-out-of-bounds writes in __do_index() and sw842_decompress() when zram read the page back.

Check the remaining output before both operations and return -ENOSPC, matching the other output-producing templates.

---
Review notes:

- Baseline i386 and x86_64 return success and change post-capacity canaries for both vectors; fixed kernels return -ENOSPC without changing them.
- QEMU 11.0.2 TCG with x86_64 KASAN reproduced both operations through zram. A 568-byte compressed page was written to a virtio backing disk with compressed_writeback enabled, then its backing block was replaced before readback. Since zram supplies a PAGE_SIZE destination, these were PAGE_SIZE-scaled versions of the KUnit streams: the baseline reported an eight-byte vmalloc-out-of-bounds write in __do_index() for the indexed copy and a one-byte write in sw842_decompress() for short data. Fixed readback returned -EIO without a KASAN report.

 lib/842/842_decompress.c | 5 +++++
 1 file changed, 5 insertions(+)

index 582085ef8b49c..87d1e0f8a4926 100644
@@ -165,6 +165,9 @@ static int __do_index(struct sw842_param *p, u8 size, u8 bits, u64 fsize)
 	u64 index, offset, total = round_down(p->out - p->ostart, 8);
 	int ret;
 
+	if (size > p->olen)
+		return -ENOSPC;
+
 	ret = next_bits(p, &index, bits);
 	if (ret)
 		return ret;

@@ -344,6 +347,8 @@ int sw842_decompress(const u8 *in, unsigned int ilen,
 
 			if (!bytes || bytes > SHORT_DATA_BITS_MAX)
 				return -EINVAL;
+			if (bytes > p.olen)
+				return -ENOSPC;
 
 			while (bytes-- > 0) {
 				ret = next_bits(&p, &tmp, 8);

2.53.0

An 842 repeat template copies the preceding eight-byte output block. The decoder rejects a repeat only when no output has been produced. A short-data operation can produce between one and seven bytes before a repeat.

Such a stream makes the repeat copy read before the start of the output buffer. Depending on the surrounding mapping, this can fault or bring preceding memory into the decompressed data.

This is also reachable through zram's compressed writeback path. With 842 selected, targeted corruption of the compressed data on its backing device to a short-data-then-repeat stream made a KASAN kernel report a vmalloc-out-of-bounds read when zram read the page back.

Require at least one complete output block before accepting a repeat.

---
Review notes:

- add_short_data_template() has one caller, immediately before add_end_template(), so valid compressor output cannot place a repeat after short data.
- QEMU 11.0.2 TCG with x86_64 KASAN reproduced this through zram. A page was written to a virtio backing disk with compressed_writeback enabled, then the backing block was replaced with SHORT_DATA(seven bytes), REPEAT(one block), END and CRC. Baseline readback reported an eight-byte vmalloc-out-of-bounds read starting one byte before zram's output page. Fixed readback returned -EIO without a KASAN report.

 lib/842/842_decompress.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 87d1e0f8a4926..45a9815abd637 100644
@@ -309,7 +309,7 @@ int sw842_decompress(const u8 *in, unsigned int ilen,
 			if (ret)
 				return ret;
 
-			if (p.out == out) /* no previous bytes */
+			if (p.out - p.ostart < 8) /* no complete previous block */
 				return -EINVAL;
 
 			/* copy rep + 1 */

2.53.0

Add table-driven tests for the 842 decompressor's output and history validation.

Three malformed streams cover an indexed copy larger than the remaining output, short data larger than the remaining output, and a repeat after fewer than eight output bytes. Every case checks the return value, output length and fixed guard bands on both sides of the output. Successful cases also check complete contents from a poison-filled destination. The repeat vector's CRC includes the leading guard byte, so the unfixed decoder successfully validates the CRC after reading before the buffer.

Three valid streams cover the same boundaries at equality: an I8 index consuming the final eight bytes, five short-data bytes consuming the final five, and a repeat with exactly one block of history.

On an unmodified baseline the malformed cases fail and the boundary cases pass; with the preceding fixes all six pass.

---
Review notes:

- CONFIG_842_DECOMPRESS_KUNIT_TEST=m selects 842_DECOMPRESS=m.
- A fixed x86_64 CONFIG_PROVE_LOCKING=y run passes all six cases without a lockdep report.

 MAINTAINERS                      |   1 +
 lib/Kconfig.debug                |  15 ++++
 lib/tests/842_decompress_kunit.c | 144 +++++++++++++++++++++++++++++++
 lib/tests/Makefile               |   1 +
 4 files changed, 161 insertions(+)
 create mode 100644 lib/tests/842_decompress_kunit.c

index 98c528c99917a..4456261cf5ea4 100644
@@ -12459,6 +12459,7 @@ F:	drivers/crypto/nx/Makefile
 F:	drivers/crypto/nx/nx-842*
 F:	include/linux/sw842.h
 F:	lib/842/
+F:	lib/tests/842_decompress_kunit.c
 
 IBM Power in-Nest Crypto Acceleration
 M:	Breno Leitão <[email protected]>

index 134b15a44625e..48201092b3df6 100644
@@ -2225,6 +2225,21 @@ menuconfig RUNTIME_TESTING_MENU
 
 if RUNTIME_TESTING_MENU
 
+config 842_DECOMPRESS_KUNIT_TEST
+	tristate "KUnit tests for the 842 decompressor" if !KUNIT_ALL_TESTS
+	depends on KUNIT
+	select 842_DECOMPRESS
+	default KUNIT_ALL_TESTS
+	help
+	  Enable stream-validation and boundary tests for the software 842
+	  decompressor. The tests exercise indexed copies, short data and
+	  repeat operations at valid and invalid output or history boundaries.
+
+	  For more information on KUnit and unit tests in general, refer to
+	  Documentation/dev-tools/kunit/.
+
+	  If unsure, say N.
+
 config TEST_DHRY
 	tristate "Dhrystone benchmark test"
 	help

index 3cac3b63a7522..89ce3acad9c39 100644
@@ -4,6 +4,7 @@
 
 # KUnit tests
 CFLAGS_bitfield_kunit.o := $(DISABLE_STRUCTLEAK_PLUGIN)
+obj-$(CONFIG_842_DECOMPRESS_KUNIT_TEST) += 842_decompress_kunit.o
 obj-$(CONFIG_BASE64_KUNIT) += base64_kunit.o
 obj-$(CONFIG_BITOPS_KUNIT) += bitops_kunit.o
 obj-$(CONFIG_BITFIELD_KUNIT) += bitfield_kunit.o

2.53.0

Hi Karl,

Thanks for your patch, which is now commit cca072cb1423191c ("lib/842: add KUnit tests for the decompressor") in crypto/master

Hi Geert,

Wow, good catch, thank you! I'll prepare a follow-up to fix it.

I also found a few other KUnit tests using the same select pattern, so I'll look into fixing those as well.

Karl

All applied. Thanks.

Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt