Skip to contentSkip to conversation

crypto: consolidate and clean up compression APIs

RFC PATCH In review Ard Biesheuvel 7 participants 46 messages Jul 18, 2023 – Aug 3, 2023

Subsystems (15)
lore
Share & tools
lore
#0 lore raw

cover crypto: consolidate and clean up compression APIs

This series is presented as an RFC, because I haven't quite convinced myself that the acomp API really needs both scatterlists and request objects to encapsulate the in- and output buffers, and perhaps there are more drastic simplifications that we might consider.

However, the current situation with comp, scomp and acomp APIs is definitely something that needs cleaning up, and so I implemented this series under the working assumption that we will keep the current acomp semantics wrt scatterlists and request objects.

Patch #1 drops zlib-deflate support in software, along with the test cases we have for it. This has no users and should have never been added.

Patch #2 removes the support for on-the-fly allocation of destination buffers and scatterlists from the Intel QAT driver. This is never used, and not even implemented by all drivers (the HiSilicon ZIP driver does not support it). The diffstat of this patch makes a good case why the caller should be in charge of allocating the memory, not the driver.

Patch #3 removes this on-the-fly allocation from the core acomp API.

Patch #4 does a minimal conversion of IPcomp to the acomp API.

Patch #5 and #6 are independent UBIFS fixes for things I ran into while working on patch #7.

Patch #7 converts UBIFS to the acomp API.

Patch #8 converts the zram block driver to the acomp API.

Patches #9 to #19 remove the existing 'comp' API implementations as well as the core plumbing, now that all clients of the API have been converted. (Note that pstore stopped using the 'comp' API as well, but these changes are already queued elsewhere)

Patch #20 converts the generic deflate compression driver to the acomp API, so that it can natively operate on discontiguous buffers, rather than requiring scratch buffers. This is the only IPcomp compression algorithm we actually implement in software in the kernel, and this conversion could help IPcomp if we decide to convert it further, and remove the code that 'linearizes' SKBs in order to present them to the compression API as a contiguous range.

Patch #21 converts the acomp-to-scomp adaptation layer so it no longer requires per-CPU scratch buffers. This takes advantage of the fact that all existing users of the acomp API pass contiguous memory regions, and so scratch buffers are only needed in exceptional cases, and can be allocated and deallocated on the fly. This removes the need for preallocated per-CPU scratch buffers that can easily add up to tens of megabytes on modern systems with high core counts and SMT.

These changes have been build tested and only lightly runtime tested. In particular, I haven't performed any thorough testing on the acomp conversions of IPcomp, UBIFS and ZRAM. Any hints on which respective methods and test cases to use here are highly appreciated.

Cc Herbert Xu <[email protected]>
Cc Eric Biggers <[email protected]>
Cc Kees Cook <[email protected]>
Cc Haren Myneni <[email protected]>
Cc Nick Terrell <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Sergey Senozhatsky <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Giovanni Cabiddu <[email protected]>
Cc Richard Weinberger <[email protected]>
Cc David Ahern <[email protected]>
Cc Eric Dumazet <[email protected]>
Cc Jakub Kicinski <[email protected]>
Cc Paolo Abeni <[email protected]>
Cc Steffen Klassert <[email protected]>

Ard Biesheuvel (21):
crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"
crypto: qat - Drop support for allocating destination buffers
crypto: acompress - Drop destination scatterlist allocation feature
net: ipcomp: Migrate to acomp API from deprecated comp API
ubifs: Pass worst-case buffer size to compression routines
ubifs: Avoid allocating buffer space unnecessarily
ubifs: Migrate to acomp compression API
zram: Migrate to acomp compression API
crypto: nx - Migrate to scomp API
crypto: 842 - drop obsolete 'comp' implementation
crypto: deflate - drop obsolete 'comp' implementation
crypto: lz4 - drop obsolete 'comp' implementation
crypto: lz4hc - drop obsolete 'comp' implementation
crypto: lzo-rle - drop obsolete 'comp' implementation
crypto: lzo - drop obsolete 'comp' implementation
crypto: zstd - drop obsolete 'comp' implementation
crypto: cavium/zip - drop obsolete 'comp' implementation
crypto: compress_null - drop obsolete 'comp' implementation
crypto: remove obsolete 'comp' compression API
crypto: deflate - implement acomp API directly
crypto: scompress - Drop the use of per-cpu scratch buffers

 Documentation/crypto/architecture.rst               |   2 -
 crypto/842.c                                        |  63 +---
 crypto/Makefile                                     |   2 +-
 crypto/acompress.c                                  |   6 -
 crypto/api.c                                        |   4 -
 crypto/compress.c                                   |  32 --
 crypto/crypto_null.c                                |  31 +-
 crypto/crypto_user_base.c                           |  16 -
 crypto/crypto_user_stat.c                           |   4 -
 crypto/deflate.c                                    | 386 ++++++--------------
 crypto/lz4.c                                        |  61 +---
 crypto/lz4hc.c                                      |  63 +---
 crypto/lzo-rle.c                                    |  60 +--
 crypto/lzo.c                                        |  60 +--
 crypto/proc.c                                       |   3 -
 crypto/scompress.c                                  | 169 ++++-----
 crypto/testmgr.c                                    | 184 +---------
 crypto/testmgr.h                                    |  75 ----
 crypto/zstd.c                                       |  56 +--
 drivers/block/zram/zcomp.c                          |  67 +++-
 drivers/block/zram/zcomp.h                          |   7 +-
 drivers/block/zram/zram_drv.c                       |  12 +-
 drivers/crypto/cavium/zip/zip_crypto.c              |  40 --
 drivers/crypto/cavium/zip/zip_crypto.h              |  10 -
 drivers/crypto/cavium/zip/zip_main.c                |  50 +--
 drivers/crypto/intel/qat/qat_common/qat_bl.c        | 159 --------
 drivers/crypto/intel/qat/qat_common/qat_bl.h        |   6 -
 drivers/crypto/intel/qat/qat_common/qat_comp_algs.c |  86 +----
 drivers/crypto/intel/qat/qat_common/qat_comp_req.h  |  10 -
 drivers/crypto/nx/nx-842.c                          |  34 +-
 drivers/crypto/nx/nx-842.h                          |  14 +-
 drivers/crypto/nx/nx-common-powernv.c               |  30 +-
 drivers/crypto/nx/nx-common-pseries.c               |  32 +-
 fs/ubifs/compress.c                                 |  61 +++-
 fs/ubifs/file.c                                     |  46 +--
 fs/ubifs/journal.c                                  |  33 +-
 fs/ubifs/ubifs.h                                    |  15 +-
 include/crypto/acompress.h                          |  21 +-
 include/crypto/internal/scompress.h                 |   2 -
 include/crypto/scatterwalk.h                        |   2 +-
 include/linux/crypto.h                              |  49 +--
 include/net/ipcomp.h                                |   4 +-
 net/xfrm/xfrm_algo.c                                |   7 +-
 net/xfrm/xfrm_ipcomp.c                              | 107 ++++--
 44 files changed, 502 insertions(+), 1679 deletions(-)
 delete mode 100644 crypto/compress.c
Signature · 1 line
2.39.2
#1 lore raw

RFC PATCH 1/21 crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"

This reverts commit a368f43d6e3a001e684e9191a27df384fbff12f5.

"zlib-deflate" was introduced 6 years ago, but it does not have any users. So let's remove the generic implementation and the test vectors, but retain the "zlib-deflate" entry in the testmgr code to avoid introducing warning messages on systems that implement zlib-deflate in hardware.

Note that RFC 1950 which forms the basis of this algorithm dates back to 1996, and predates RFC 1951, on which the existing IPcomp is based and which we have supported in the kernel since 2003. So it seems rather unlikely that we will ever grow the need to support zlib-deflate.

---
 crypto/deflate.c | 61 +++++-----------
 crypto/testmgr.c |  8 +--
 crypto/testmgr.h | 75 --------------------
 3 files changed, 18 insertions(+), 126 deletions(-)
crypto/deflate.c +17 -44
index b2a46f6dc961e71d..f4f127078fe2a5aa 100644
@@ -39,24 +39,20 @@ struct deflate_ctx {
 	struct z_stream_s decomp_stream;
 };
 
-static int deflate_comp_init(struct deflate_ctx *ctx, int format)
+static int deflate_comp_init(struct deflate_ctx *ctx)
 {
 	int ret = 0;
 	struct z_stream_s *stream = &ctx->comp_stream;
 
 	stream->workspace = vzalloc(zlib_deflate_workspacesize(
-				    MAX_WBITS, MAX_MEM_LEVEL));
+				-DEFLATE_DEF_WINBITS, DEFLATE_DEF_MEMLEVEL));
 	if (!stream->workspace) {
 		ret = -ENOMEM;
 		goto out;
 	}
-	if (format)
-		ret = zlib_deflateInit(stream, 3);
-	else
-		ret = zlib_deflateInit2(stream, DEFLATE_DEF_LEVEL, Z_DEFLATED,
-					-DEFLATE_DEF_WINBITS,
-					DEFLATE_DEF_MEMLEVEL,
-					Z_DEFAULT_STRATEGY);
+	ret = zlib_deflateInit2(stream, DEFLATE_DEF_LEVEL, Z_DEFLATED,
+	                        -DEFLATE_DEF_WINBITS, DEFLATE_DEF_MEMLEVEL,
+	                        Z_DEFAULT_STRATEGY);
 	if (ret != Z_OK) {
 		ret = -EINVAL;
 		goto out_free;
@@ -68,7 +64,7 @@ static int deflate_comp_init(struct deflate_ctx *ctx, int format)
 	goto out;
 }
 
-static int deflate_decomp_init(struct deflate_ctx *ctx, int format)
+static int deflate_decomp_init(struct deflate_ctx *ctx)
 {
 	int ret = 0;
 	struct z_stream_s *stream = &ctx->decomp_stream;
@@ -78,10 +74,7 @@ static int deflate_decomp_init(struct deflate_ctx *ctx, int format)
 		ret = -ENOMEM;
 		goto out;
 	}
-	if (format)
-		ret = zlib_inflateInit(stream);
-	else
-		ret = zlib_inflateInit2(stream, -DEFLATE_DEF_WINBITS);
+	ret = zlib_inflateInit2(stream, -DEFLATE_DEF_WINBITS);
 	if (ret != Z_OK) {
 		ret = -EINVAL;
 		goto out_free;
@@ -105,21 +98,21 @@ static void deflate_decomp_exit(struct deflate_ctx *ctx)
 	vfree(ctx->decomp_stream.workspace);
 }
 
-static int __deflate_init(void *ctx, int format)
+static int __deflate_init(void *ctx)
 {
 	int ret;
 
-	ret = deflate_comp_init(ctx, format);
+	ret = deflate_comp_init(ctx);
 	if (ret)
 		goto out;
-	ret = deflate_decomp_init(ctx, format);
+	ret = deflate_decomp_init(ctx);
 	if (ret)
 		deflate_comp_exit(ctx);
 out:
 	return ret;
 }
 
-static void *gen_deflate_alloc_ctx(struct crypto_scomp *tfm, int format)
+static void *deflate_alloc_ctx(struct crypto_scomp *tfm)
 {
 	struct deflate_ctx *ctx;
 	int ret;
@@ -128,7 +121,7 @@ static void *gen_deflate_alloc_ctx(struct crypto_scomp *tfm, int format)
 	if (!ctx)
 		return ERR_PTR(-ENOMEM);
 
-	ret = __deflate_init(ctx, format);
+	ret = __deflate_init(ctx);
 	if (ret) {
 		kfree(ctx);
 		return ERR_PTR(ret);
@@ -137,21 +130,11 @@ static void *gen_deflate_alloc_ctx(struct crypto_scomp *tfm, int format)
 	return ctx;
 }
 
-static void *deflate_alloc_ctx(struct crypto_scomp *tfm)
-{
-	return gen_deflate_alloc_ctx(tfm, 0);
-}
-
-static void *zlib_deflate_alloc_ctx(struct crypto_scomp *tfm)
-{
-	return gen_deflate_alloc_ctx(tfm, 1);
-}
-
 static int deflate_init(struct crypto_tfm *tfm)
 {
 	struct deflate_ctx *ctx = crypto_tfm_ctx(tfm);
 
-	return __deflate_init(ctx, 0);
+	return __deflate_init(ctx);
 }
 
 static void __deflate_exit(void *ctx)
@@ -286,7 +269,7 @@ static struct crypto_alg alg = {
 	.coa_decompress  	= deflate_decompress } }
 };
 
-static struct scomp_alg scomp[] = { {
+static struct scomp_alg scomp = {
 	.alloc_ctx		= deflate_alloc_ctx,
 	.free_ctx		= deflate_free_ctx,
 	.compress		= deflate_scompress,
@@ -296,17 +279,7 @@ static struct scomp_alg scomp[] = { {
 		.cra_driver_name = "deflate-scomp",
 		.cra_module	 = THIS_MODULE,
 	}
-}, {
-	.alloc_ctx		= zlib_deflate_alloc_ctx,
-	.free_ctx		= deflate_free_ctx,
-	.compress		= deflate_scompress,
-	.decompress		= deflate_sdecompress,
-	.base			= {
-		.cra_name	= "zlib-deflate",
-		.cra_driver_name = "zlib-deflate-scomp",
-		.cra_module	 = THIS_MODULE,
-	}
-} };
+};
 
 static int __init deflate_mod_init(void)
 {
@@ -316,7 +289,7 @@ static int __init deflate_mod_init(void)
 	if (ret)
 		return ret;
 
-	ret = crypto_register_scomps(scomp, ARRAY_SIZE(scomp));
+	ret = crypto_register_scomp(&scomp);
 	if (ret) {
 		crypto_unregister_alg(&alg);
 		return ret;
@@ -328,7 +301,7 @@ static int __init deflate_mod_init(void)
 static void __exit deflate_mod_fini(void)
 {
 	crypto_unregister_alg(&alg);
-	crypto_unregister_scomps(scomp, ARRAY_SIZE(scomp));
+	crypto_unregister_scomp(&scomp);
 }
 
 subsys_initcall(deflate_mod_init);
crypto/testmgr.c +1 -7
index 216878c8bc3d62f8..b41a8e8c1d1a1987 100644
@@ -5774,14 +5774,8 @@ static const struct alg_test_desc alg_test_descs[] = {
 		}
 	}, {
 		.alg = "zlib-deflate",
-		.test = alg_test_comp,
+		.test = alg_test_null,
 		.fips_allowed = 1,
-		.suite = {
-			.comp = {
-				.comp = __VECS(zlib_deflate_comp_tv_template),
-				.decomp = __VECS(zlib_deflate_decomp_tv_template)
-			}
-		}
 	}, {
 		.alg = "zstd",
 		.test = alg_test_comp,
crypto/testmgr.h +0 -75

81 lines not on this page · raw message

Signature · 1 line
2.39.2
#2 lore raw

Re: [RFC PATCH 01/21] crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"

Quoted 13 lines · Ard Biesheuvel

This reverts commit a368f43d6e3a001e684e9191a27df384fbff12f5.

"zlib-deflate" was introduced 6 years ago, but it does not have any users. So let's remove the generic implementation and the test vectors, but retain the "zlib-deflate" entry in the testmgr code to avoid introducing warning messages on systems that implement zlib-deflate in hardware.

Note that RFC 1950 which forms the basis of this algorithm dates back to 1996, and predates RFC 1951, on which the existing IPcomp is based and which we have supported in the kernel since 2003. So it seems rather unlikely that we will ever grow the need to support zlib-deflate.

Quoted 6 lines
---
 crypto/deflate.c | 61 +++++-----------
 crypto/testmgr.c |  8 +--
 crypto/testmgr.h | 75 --------------------
 3 files changed, 18 insertions(+), 126 deletions(-)

So if this is really unused, it's probably fair to remove it on that basis. However, it's not correct to claim that DEFLATE is obsoleted by zlib (the data format). zlib is just DEFLATE plus a checksum, as is gzip.

Many users of zlib or gzip use an external checksum and therefore would be better served by DEFLATE, avoiding a redundant builtin checksum. Typically, people have chosen zlib or gzip simply because their compression library defaulted to it, they didn't understand the difference, and they overlooked that they're paying the price for a redundant builtin checksum.

An example of someone doing it right is EROFS, which is working on adding DEFLATE support (not zlib or gzip!):
https://lore.kernel.org/r/[email protected]

Of course, they are using the library API instead of the clumsy crypto API.

- Eric

#3 lore raw

Re: [RFC PATCH 01/21] crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"

Quoted 31 lines · Eric Biggers
Quoted 13 lines · Ard Biesheuvel

This reverts commit a368f43d6e3a001e684e9191a27df384fbff12f5.

"zlib-deflate" was introduced 6 years ago, but it does not have any users. So let's remove the generic implementation and the test vectors, but retain the "zlib-deflate" entry in the testmgr code to avoid introducing warning messages on systems that implement zlib-deflate in hardware.

Note that RFC 1950 which forms the basis of this algorithm dates back to 1996, and predates RFC 1951, on which the existing IPcomp is based and which we have supported in the kernel since 2003. So it seems rather unlikely that we will ever grow the need to support zlib-deflate.

Quoted 6 lines
---
 crypto/deflate.c | 61 +++++-----------
 crypto/testmgr.c |  8 +--
 crypto/testmgr.h | 75 --------------------
 3 files changed, 18 insertions(+), 126 deletions(-)

So if this is really unused, it's probably fair to remove it on that basis. However, it's not correct to claim that DEFLATE is obsoleted by zlib (the data format). zlib is just DEFLATE plus a checksum, as is gzip.

Many users of zlib or gzip use an external checksum and therefore would be better served by DEFLATE, avoiding a redundant builtin checksum. Typically, people have chosen zlib or gzip simply because their compression library defaulted to it, they didn't understand the difference, and they overlooked that they're paying the price for a redundant builtin checksum.

Quoted 6 lines

An example of someone doing it right is EROFS, which is working on adding DEFLATE support (not zlib or gzip!):
https://lore.kernel.org/r/[email protected]

Of course, they are using the library API instead of the clumsy crypto API.

Ah, I misread this patch, sorry. It's actually removing support for zlib (the data format) from the scomp API, leaving just DEFLATE. That's fine too; again, it ultimately just depends on what is actually being used via the scomp API. But similarly you can't really claim that zlib is obsoleted by DEFLATE just because of the RFC dates. As I mentioned, many people do use zlib (the data format), often just because it's the default of zlib (the library) and they didn't know any better. For example, btrfs compression supports zlib.

- Eric

#4 lore raw

Re: [RFC PATCH 01/21] crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"

Quoted 40 lines · Eric Biggers
Quoted 31 lines · Eric Biggers
Quoted 13 lines · Ard Biesheuvel

This reverts commit a368f43d6e3a001e684e9191a27df384fbff12f5.

"zlib-deflate" was introduced 6 years ago, but it does not have any users. So let's remove the generic implementation and the test vectors, but retain the "zlib-deflate" entry in the testmgr code to avoid introducing warning messages on systems that implement zlib-deflate in hardware.

Note that RFC 1950 which forms the basis of this algorithm dates back to 1996, and predates RFC 1951, on which the existing IPcomp is based and which we have supported in the kernel since 2003. So it seems rather unlikely that we will ever grow the need to support zlib-deflate.

Quoted 6 lines
---
 crypto/deflate.c | 61 +++++-----------
 crypto/testmgr.c |  8 +--
 crypto/testmgr.h | 75 --------------------
 3 files changed, 18 insertions(+), 126 deletions(-)

So if this is really unused, it's probably fair to remove it on that basis. However, it's not correct to claim that DEFLATE is obsoleted by zlib (the data format). zlib is just DEFLATE plus a checksum, as is gzip.

Many users of zlib or gzip use an external checksum and therefore would be better served by DEFLATE, avoiding a redundant builtin checksum. Typically, people have chosen zlib or gzip simply because their compression library defaulted to it, they didn't understand the difference, and they overlooked that they're paying the price for a redundant builtin checksum.

Quoted 6 lines

An example of someone doing it right is EROFS, which is working on adding DEFLATE support (not zlib or gzip!):
https://lore.kernel.org/r/[email protected]

Of course, they are using the library API instead of the clumsy crypto API.

Quoted 8 lines

Ah, I misread this patch, sorry. It's actually removing support for zlib (the data format) from the scomp API, leaving just DEFLATE. That's fine too; again, it ultimately just depends on what is actually being used via the scomp API. But similarly you can't really claim that zlib is obsoleted by DEFLATE just because of the RFC dates. As I mentioned, many people do use zlib (the data format), often just because it's the default of zlib (the library) and they didn't know any better. For example, btrfs compression supports zlib.

I am not suggesting either is obsolete. I am merely pointing out that zlib-deflate is as old as plain deflate, and so we could have implemented both at the same time when IPcomp support was added, but we never bothered.

#5 lore raw

Re: [RFC PATCH 01/21] crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"

In reply to Ard Biesheuvel
Quoted 41 lines · Ard Biesheuvel

This reverts commit a368f43d6e3a001e684e9191a27df384fbff12f5.

"zlib-deflate" was introduced 6 years ago, but it does not have any users. So let's remove the generic implementation and the test vectors, but retain the "zlib-deflate" entry in the testmgr code to avoid introducing warning messages on systems that implement zlib-deflate in hardware.

Note that RFC 1950 which forms the basis of this algorithm dates back to 1996, and predates RFC 1951, on which the existing IPcomp is based and which we have supported in the kernel since 2003. So it seems rather unlikely that we will ever grow the need to support zlib-deflate.

---
 crypto/deflate.c | 61 +++++-----------
 crypto/testmgr.c |  8 +--
 crypto/testmgr.h | 75 --------------------
 3 files changed, 18 insertions(+), 126 deletions(-)
crypto/deflate.c +2 -2
index b2a46f6dc961e71d..f4f127078fe2a5aa 100644
@@ -39,24 +39,20 @@ struct deflate_ctx {
 	struct z_stream_s decomp_stream;
 };
 
-static int deflate_comp_init(struct deflate_ctx *ctx, int format)
+static int deflate_comp_init(struct deflate_ctx *ctx)
 {
 	int ret = 0;
 	struct z_stream_s *stream = &ctx->comp_stream;
 
 	stream->workspace = vzalloc(zlib_deflate_workspacesize(
-				    MAX_WBITS, MAX_MEM_LEVEL));
+				-DEFLATE_DEF_WINBITS, DEFLATE_DEF_MEMLEVEL));
 	if (!stream->workspace) {
 		ret = -ENOMEM;
 		goto out;
 	}

crypto/deflate.c struct deflate_ctx { · line 55

Quoted 10 lines
crypto/deflate.c +3 -7
-	if (format)
-		ret = zlib_deflateInit(stream, 3);
-	else
-		ret = zlib_deflateInit2(stream, DEFLATE_DEF_LEVEL, Z_DEFLATED,
-					-DEFLATE_DEF_WINBITS,
-					DEFLATE_DEF_MEMLEVEL,
-					Z_DEFAULT_STRATEGY);
+	ret = zlib_deflateInit2(stream, DEFLATE_DEF_LEVEL, Z_DEFLATED,
+	                        -DEFLATE_DEF_WINBITS, DEFLATE_DEF_MEMLEVEL,
+	                        Z_DEFAULT_STRATEGY);

nit: The two lines above partially use spaces instead of tabs for
indentation.

...

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

maintainer
#6 lore raw

Re: [RFC PATCH 01/21] crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"

In reply to Ard Biesheuvel

Hi Ard,

Quoted 8 lines · Ard Biesheuvel

This reverts commit a368f43d6e3a001e684e9191a27df384fbff12f5.

"zlib-deflate" was introduced 6 years ago, but it does not have any users. So let's remove the generic implementation and the test vectors, but retain the "zlib-deflate" entry in the testmgr code to avoid introducing warning messages on systems that implement zlib-deflate in hardware.

Quoted 6 lines

Note that RFC 1950 which forms the basis of this algorithm dates back to 1996, and predates RFC 1951, on which the existing IPcomp is based and which we have supported in the kernel since 2003. So it seems rather unlikely that we will ever grow the need to support zlib-deflate.

Support for zlib-deflate was added for [1] but that work was not completed.

Based on [2], either we leave this SW implementation or we remove the HW implementations in the QAT [3] and in the Hisilicon Zip [4] drivers.

[1] https://patchwork.kernel.org/project/linux-btrfs/patch/[email protected]/
[2] https://lore.kernel.org/lkml/ZIw%[email protected]/
[3] https://elixir.bootlin.com/linux/latest/source/drivers/crypto/intel/qat/qat_common/qat_comp_algs.c#L457
[4] https://elixir.bootlin.com/linux/latest/source/drivers/crypto/hisilicon/zip/zip_crypto.c#L754

Regards,

Signature · 1 line
Giovanni
#7 lore raw

Re: [RFC PATCH 01/21] crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"

Hello Giovanni,

Quoted 12 lines · Giovanni Cabiddu

Hi Ard,

Quoted 8 lines · Ard Biesheuvel

This reverts commit a368f43d6e3a001e684e9191a27df384fbff12f5.

"zlib-deflate" was introduced 6 years ago, but it does not have any users. So let's remove the generic implementation and the test vectors, but retain the "zlib-deflate" entry in the testmgr code to avoid introducing warning messages on systems that implement zlib-deflate in hardware.

Quoted 9 lines
Quoted 6 lines

Note that RFC 1950 which forms the basis of this algorithm dates back to 1996, and predates RFC 1951, on which the existing IPcomp is based and which we have supported in the kernel since 2003. So it seems rather unlikely that we will ever grow the need to support zlib-deflate.

Support for zlib-deflate was added for [1] but that work was not completed.

Any clue why zlib_deflate was chosen in this case?

/me also notes that this is another occurrence of the antipattern where we use an asynchronous API and subsequently sleep on the completion.

Quoted 3 lines

Based on [2], either we leave this SW implementation or we remove the HW implementations in the QAT [3] and in the Hisilicon Zip [4] drivers.

That would work for me as well - dead code is just busywork.

Quoted 9 lines

[1] https://patchwork.kernel.org/project/linux-btrfs/patch/[email protected]/
[2] https://lore.kernel.org/lkml/ZIw%[email protected]/
[3] https://elixir.bootlin.com/linux/latest/source/drivers/crypto/intel/qat/qat_common/qat_comp_algs.c#L457
[4] https://elixir.bootlin.com/linux/latest/source/drivers/crypto/hisilicon/zip/zip_crypto.c#L754

Regards,

--
Giovanni

maintainer
#8 lore raw

Re: [RFC PATCH 01/21] crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"

Quoted 1 line · Ard Biesheuvel

Any clue why zlib_deflate was chosen in this case?

I believe it was done for integrity.
The output of zlib_deflate is raw deflate with an header and a footer defined in rfc1950. The footer contains a checksum which gets calculated over the uncompressed data during compression and checked after decompression.

Signature · 1 line
Giovanni
#9 lore raw

RFC PATCH 2/21 crypto: qat - Drop support for allocating destination buffers

In reply to Ard Biesheuvel

Remove the logic that allocates the destination scatterlist and backing pages on the fly when no destination is provided: this is a rather dubious proposition, given that the caller is in a far better position to estimate the size of such a buffer, or how it should be allocated.

This feature has no current users, so let's remove it while we still can.

---
 drivers/crypto/intel/qat/qat_common/qat_bl.c        | 159 --------------------
 drivers/crypto/intel/qat/qat_common/qat_bl.h        |   6 -
 drivers/crypto/intel/qat/qat_common/qat_comp_algs.c |  86 +----------
 drivers/crypto/intel/qat/qat_common/qat_comp_req.h  |  10 --
 4 files changed, 1 insertion(+), 260 deletions(-)
drivers/crypto/intel/qat/qat_common/qat_bl.c +0 -159
index 76baed0a76c0ee93..94f6a5fe0f3dea75 100644
@@ -249,162 +249,3 @@ int qat_bl_sgl_to_bufl(struct adf_accel_dev *accel_dev,
 				    extra_dst_buff, sz_extra_dst_buff,
 				    sskip, dskip, flags);
 }
-
-static void qat_bl_sgl_unmap(struct adf_accel_dev *accel_dev,
-			     struct qat_alg_buf_list *bl)
-{
-	struct device *dev = &GET_DEV(accel_dev);
-	int n = bl->num_bufs;
-	int i;
-
-	for (i = 0; i < n; i++)
-		if (!dma_mapping_error(dev, bl->buffers[i].addr))
-			dma_unmap_single(dev, bl->buffers[i].addr,
-					 bl->buffers[i].len, DMA_FROM_DEVICE);
-}
-
-static int qat_bl_sgl_map(struct adf_accel_dev *accel_dev,
-			  struct scatterlist *sgl,
-			  struct qat_alg_buf_list **bl)
-{
-	struct device *dev = &GET_DEV(accel_dev);
-	struct qat_alg_buf_list *bufl;
-	int node = dev_to_node(dev);
-	struct scatterlist *sg;
-	int n, i, sg_nctr;
-	size_t sz;
-
-	n = sg_nents(sgl);
-	sz = struct_size(bufl, buffers, n);
-	bufl = kzalloc_node(sz, GFP_KERNEL, node);
-	if (unlikely(!bufl))
-		return -ENOMEM;
-
-	for (i = 0; i < n; i++)
-		bufl->buffers[i].addr = DMA_MAPPING_ERROR;
-
-	sg_nctr = 0;
-	for_each_sg(sgl, sg, n, i) {
-		int y = sg_nctr;
-
-		if (!sg->length)
-			continue;
-
-		bufl->buffers[y].addr = dma_map_single(dev, sg_virt(sg),
-						       sg->length,
-						       DMA_FROM_DEVICE);
-		bufl->buffers[y].len = sg->length;
-		if (unlikely(dma_mapping_error(dev, bufl->buffers[y].addr)))
-			goto err_map;
-		sg_nctr++;
-	}
-	bufl->num_bufs = sg_nctr;
-	bufl->num_mapped_bufs = sg_nctr;
-
-	*bl = bufl;
-
-	return 0;
-
-err_map:
-	for (i = 0; i < n; i++)
-		if (!dma_mapping_error(dev, bufl->buffers[i].addr))
-			dma_unmap_single(dev, bufl->buffers[i].addr,
-					 bufl->buffers[i].len,
-					 DMA_FROM_DEVICE);
-	kfree(bufl);
-	*bl = NULL;
-
-	return -ENOMEM;
-}
-
-static void qat_bl_sgl_free_unmap(struct adf_accel_dev *accel_dev,
-				  struct scatterlist *sgl,
-				  struct qat_alg_buf_list *bl,
-				  bool free_bl)
-{
-	if (bl) {
-		qat_bl_sgl_unmap(accel_dev, bl);
-
-		if (free_bl)
-			kfree(bl);
-	}
-	if (sgl)
-		sgl_free(sgl);
-}
-
-static int qat_bl_sgl_alloc_map(struct adf_accel_dev *accel_dev,
-				struct scatterlist **sgl,
-				struct qat_alg_buf_list **bl,
-				unsigned int dlen,
-				gfp_t gfp)
-{
-	struct scatterlist *dst;
-	int ret;
-
-	dst = sgl_alloc(dlen, gfp, NULL);
-	if (!dst) {
-		dev_err(&GET_DEV(accel_dev), "sg_alloc failed\n");
-		return -ENOMEM;
-	}
-
-	ret = qat_bl_sgl_map(accel_dev, dst, bl);
-	if (ret)
-		goto err;
-
-	*sgl = dst;
-
-	return 0;
-
-err:
-	sgl_free(dst);
-	*sgl = NULL;
-	return ret;
-}
-
-int qat_bl_realloc_map_new_dst(struct adf_accel_dev *accel_dev,
-			       struct scatterlist **sg,
-			       unsigned int dlen,
-			       struct qat_request_buffs *qat_bufs,
-			       gfp_t gfp)
-{
-	struct device *dev = &GET_DEV(accel_dev);
-	dma_addr_t new_blp = DMA_MAPPING_ERROR;
-	struct qat_alg_buf_list *new_bl;
-	struct scatterlist *new_sg;
-	size_t new_bl_size;
-	int ret;
-
-	ret = qat_bl_sgl_alloc_map(accel_dev, &new_sg, &new_bl, dlen, gfp);
-	if (ret)
-		return ret;
-
-	new_bl_size = struct_size(new_bl, buffers, new_bl->num_bufs);
-
-	/* Map new firmware SGL descriptor */
-	new_blp = dma_map_single(dev, new_bl, new_bl_size, DMA_TO_DEVICE);
-	if (unlikely(dma_mapping_error(dev, new_blp)))
-		goto err;
-
-	/* Unmap old firmware SGL descriptor */
-	dma_unmap_single(dev, qat_bufs->bloutp, qat_bufs->sz_out, DMA_TO_DEVICE);
-
-	/* Free and unmap old scatterlist */
-	qat_bl_sgl_free_unmap(accel_dev, *sg, qat_bufs->blout,
-			      !qat_bufs->sgl_dst_valid);
-
-	qat_bufs->sgl_dst_valid = false;
-	qat_bufs->blout = new_bl;
-	qat_bufs->bloutp = new_blp;
-	qat_bufs->sz_out = new_bl_size;
-
-	*sg = new_sg;
-
-	return 0;
-err:
-	qat_bl_sgl_free_unmap(accel_dev, new_sg, new_bl, true);
-
-	if (!dma_mapping_error(dev, new_blp))
-		dma_unmap_single(dev, new_blp, new_bl_size, DMA_TO_DEVICE);
-
-	return -ENOMEM;
-}
drivers/crypto/intel/qat/qat_common/qat_bl.h +0 -6
index d87e4f35ac395c76..a508d795113116c0 100644
@@ -60,10 +60,4 @@ static inline gfp_t qat_algs_alloc_flags(struct crypto_async_request *req)
 	return req->flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL : GFP_ATOMIC;
 }
 
-int qat_bl_realloc_map_new_dst(struct adf_accel_dev *accel_dev,
-			       struct scatterlist **newd,
-			       unsigned int dlen,
-			       struct qat_request_buffs *qat_bufs,
-			       gfp_t gfp);
-
 #endif
drivers/crypto/intel/qat/qat_common/qat_comp_algs.c +1 -85

130 lines not on this page · raw message

drivers/crypto/intel/qat/qat_common/qat_comp_req.h +0 -10
index 404e32c5e77838df..18a1f33a6db98e8c 100644
@@ -25,16 +25,6 @@ static inline void qat_comp_create_req(void *ctx, void *req, u64 src, u32 slen,
 	req_pars->out_buffer_sz = dlen;
 }
 
-static inline void qat_comp_override_dst(void *req, u64 dst, u32 dlen)
-{
-	struct icp_qat_fw_comp_req *fw_req = req;
-	struct icp_qat_fw_comp_req_params *req_pars = &fw_req->comp_pars;
-
-	fw_req->comn_mid.dest_data_addr = dst;
-	fw_req->comn_mid.dst_length = dlen;
-	req_pars->out_buffer_sz = dlen;
-}
-
 static inline void qat_comp_create_compression_req(void *ctx, void *req,
 						   u64 src, u32 slen,
 						   u64 dst, u32 dlen,
Signature · 1 line
2.39.2
#10 lore raw

RFC PATCH 3/21 crypto: acompress - Drop destination scatterlist allocation feature

In reply to Ard Biesheuvel

The acomp crypto code will allocate a destination scatterlist and its backing pages on the fly if no destination is passed. This feature is not used, and given that the caller should own this memory, it is far better if the caller allocates it. This is especially true for decompression, where the output size is essentially unbounded, and so the caller already needs to provide the size for this feature to work reliably.

---
 crypto/acompress.c         |  6 ----
 crypto/scompress.c         | 14 +---------
 crypto/testmgr.c           | 29 --------------------
 include/crypto/acompress.h | 16 ++---------
 4 files changed, 4 insertions(+), 61 deletions(-)
crypto/acompress.c +0 -6
index 1c682810a484dcdf..431876b0ee2096fd 100644
@@ -71,7 +71,6 @@ static int crypto_acomp_init_tfm(struct crypto_tfm *tfm)
 
 	acomp->compress = alg->compress;
 	acomp->decompress = alg->decompress;
-	acomp->dst_free = alg->dst_free;
 	acomp->reqsize = alg->reqsize;
 
 	if (alg->exit)
@@ -173,11 +172,6 @@ void acomp_request_free(struct acomp_req *req)
 	if (tfm->__crt_alg->cra_type != &crypto_acomp_type)
 		crypto_acomp_scomp_free_ctx(req);
 
-	if (req->flags & CRYPTO_ACOMP_ALLOC_OUTPUT) {
-		acomp->dst_free(req->dst);
-		req->dst = NULL;
-	}
-
 	__acomp_request_free(req);
 }
 EXPORT_SYMBOL_GPL(acomp_request_free);
crypto/scompress.c +1 -13
index 442a82c9de7def1f..3155cdce9116e092 100644
@@ -122,12 +122,9 @@ static int scomp_acomp_comp_decomp(struct acomp_req *req, int dir)
 	if (!req->src || !req->slen || req->slen > SCOMP_SCRATCH_SIZE)
 		return -EINVAL;
 
-	if (req->dst && !req->dlen)
+	if (!req->dst || !req->dlen || req->dlen > SCOMP_SCRATCH_SIZE)
 		return -EINVAL;
 
-	if (!req->dlen || req->dlen > SCOMP_SCRATCH_SIZE)
-		req->dlen = SCOMP_SCRATCH_SIZE;
-
 	scratch = raw_cpu_ptr(&scomp_scratch);
 	spin_lock(&scratch->lock);
 
@@ -139,17 +136,9 @@ static int scomp_acomp_comp_decomp(struct acomp_req *req, int dir)
 		ret = crypto_scomp_decompress(scomp, scratch->src, req->slen,
 					      scratch->dst, &req->dlen, *ctx);
 	if (!ret) {
-		if (!req->dst) {
-			req->dst = sgl_alloc(req->dlen, GFP_ATOMIC, NULL);
-			if (!req->dst) {
-				ret = -ENOMEM;
-				goto out;
-			}
-		}
 		scatterwalk_map_and_copy(scratch->dst, req->dst, 0, req->dlen,
 					 1);
 	}
-out:
 	spin_unlock(&scratch->lock);
 	return ret;
 }
@@ -197,7 +186,6 @@ int crypto_init_scomp_ops_async(struct crypto_tfm *tfm)
 
 	crt->compress = scomp_acomp_compress;
 	crt->decompress = scomp_acomp_decompress;
-	crt->dst_free = sgl_free;
 	crt->reqsize = sizeof(void *);
 
 	return 0;
crypto/testmgr.c +0 -29
index b41a8e8c1d1a1987..4971351f55dbabb9 100644
@@ -3497,21 +3497,6 @@ static int test_acomp(struct crypto_acomp *tfm,
 			goto out;
 		}
 
-#ifdef CONFIG_CRYPTO_MANAGER_EXTRA_TESTS
-		crypto_init_wait(&wait);
-		sg_init_one(&src, input_vec, ilen);
-		acomp_request_set_params(req, &src, NULL, ilen, 0);
-
-		ret = crypto_wait_req(crypto_acomp_compress(req), &wait);
-		if (ret) {
-			pr_err("alg: acomp: compression failed on NULL dst buffer test %d for %s: ret=%d\n",
-			       i + 1, algo, -ret);
-			kfree(input_vec);
-			acomp_request_free(req);
-			goto out;
-		}
-#endif
-
 		kfree(input_vec);
 		acomp_request_free(req);
 	}
@@ -3573,20 +3558,6 @@ static int test_acomp(struct crypto_acomp *tfm,
 			goto out;
 		}
 
-#ifdef CONFIG_CRYPTO_MANAGER_EXTRA_TESTS
-		crypto_init_wait(&wait);
-		acomp_request_set_params(req, &src, NULL, ilen, 0);
-
-		ret = crypto_wait_req(crypto_acomp_decompress(req), &wait);
-		if (ret) {
-			pr_err("alg: acomp: decompression failed on NULL dst buffer test %d for %s: ret=%d\n",
-			       i + 1, algo, -ret);
-			kfree(input_vec);
-			acomp_request_free(req);
-			goto out;
-		}
-#endif
-
 		kfree(input_vec);
 		acomp_request_free(req);
 	}
include/crypto/acompress.h +3 -13
index 574cffc90730f5f3..ccb6f3279bc8b32e 100644
@@ -43,15 +43,12 @@ struct acomp_req {
  *
  * @compress:		Function performs a compress operation
  * @decompress:		Function performs a de-compress operation
- * @dst_free:		Frees destination buffer if allocated inside the
- *			algorithm
  * @reqsize:		Context size for (de)compression requests
  * @base:		Common crypto API algorithm data structure
  */
 struct crypto_acomp {
 	int (*compress)(struct acomp_req *req);
 	int (*decompress)(struct acomp_req *req);
-	void (*dst_free)(struct scatterlist *dst);
 	unsigned int reqsize;
 	struct crypto_tfm base;
 };
@@ -222,8 +219,7 @@ static inline void acomp_request_set_callback(struct acomp_req *req,
 {
 	req->base.complete = cmpl;
 	req->base.data = data;
-	req->base.flags &= CRYPTO_ACOMP_ALLOC_OUTPUT;
-	req->base.flags |= flgs & ~CRYPTO_ACOMP_ALLOC_OUTPUT;
+	req->base.flags = flgs;
 }
 
 /**
@@ -233,11 +229,9 @@ static inline void acomp_request_set_callback(struct acomp_req *req,
  *
  * @req:	asynchronous compress request
  * @src:	pointer to input buffer scatterlist
- * @dst:	pointer to output buffer scatterlist. If this is NULL, the
- *		acomp layer will allocate the output memory
+ * @dst:	pointer to output buffer scatterlist
  * @slen:	size of the input buffer
- * @dlen:	size of the output buffer. If dst is NULL, this can be used by
- *		the user to specify the maximum amount of memory to allocate
+ * @dlen:	size of the output buffer
  */
 static inline void acomp_request_set_params(struct acomp_req *req,
 					    struct scatterlist *src,
@@ -249,10 +243,6 @@ static inline void acomp_request_set_params(struct acomp_req *req,
 	req->dst = dst;
 	req->slen = slen;
 	req->dlen = dlen;
-
-	req->flags &= ~CRYPTO_ACOMP_ALLOC_OUTPUT;
-	if (!req->dst)
-		req->flags |= CRYPTO_ACOMP_ALLOC_OUTPUT;
 }
 
 static inline struct crypto_istat_compress *comp_get_stat(
Signature · 1 line
2.39.2
#11 lore raw

RFC PATCH 4/21 net: ipcomp: Migrate to acomp API from deprecated comp API

In reply to Ard Biesheuvel

Migrate the IPcomp network compression code to the acomp API, in order to drop the dependency on the obsolete 'comp' API which is going away.

For the time being, this is a rather mechanical conversion replacing each comp TFM object with an acomp TFM/request object pair - this is necessary because, at this point, there is still a 1:1 relation between acomp tranforms and requests in the acomp-to-scomp adaptation layer, and this deviates from the model used by AEADs and skciphers where the TFM is fully reentrant, and operations using the same encryption keys can be issued in parallel using individual request objects but the same TFM.

Also, this minimal conversion does not yet take advantage of the fact that the acomp API takes scatterlists as input and output descriptors, which in principle removes the need to linearize the SKBs. However, given that compression code generally requires in- and output buffers to be non-overlapping, scratch buffers will always be needed, and so whether this conversion is worth while is TBD.

---
 include/crypto/acompress.h |   5 +
 include/net/ipcomp.h       |   4 +-
 net/xfrm/xfrm_algo.c       |   7 +-
 net/xfrm/xfrm_ipcomp.c     | 107 +++++++++++++-------
 4 files changed, 79 insertions(+), 44 deletions(-)
include/crypto/acompress.h +5 -0
index ccb6f3279bc8b32e..3f54e3d8815a9d0d 100644
@@ -318,4 +318,9 @@ static inline int crypto_acomp_decompress(struct acomp_req *req)
 	return crypto_comp_errstat(alg, tfm->decompress(req));
 }
 
+static inline const char *crypto_acomp_name(struct crypto_acomp *acomp)
+{
+       return crypto_tfm_alg_name(crypto_acomp_tfm(acomp));
+}
+
 #endif
include/net/ipcomp.h +2 -2
index 8660a2a6d1fc76a7..bf27ac7e3ca952e2 100644
@@ -7,12 +7,12 @@
 
 #define IPCOMP_SCRATCH_SIZE     65400
 
-struct crypto_comp;
+struct acomp_req;
 struct ip_comp_hdr;
 
 struct ipcomp_data {
 	u16 threshold;
-	struct crypto_comp * __percpu *tfms;
+	struct acomp_req * __percpu *reqs;
 };
 
 struct ip_comp_hdr;
net/xfrm/xfrm_algo.c +4 -3
index 094734fbec967505..ca411bcebc53ad4f 100644
@@ -5,6 +5,7 @@
  * Copyright (c) 2002 James Morris <[email protected]>
  */
 
+#include <crypto/acompress.h>
 #include <crypto/hash.h>
 #include <crypto/skcipher.h>
 #include <linux/module.h>
@@ -674,7 +675,7 @@ static const struct xfrm_algo_list xfrm_ealg_list = {
 static const struct xfrm_algo_list xfrm_calg_list = {
 	.algs = calg_list,
 	.entries = ARRAY_SIZE(calg_list),
-	.type = CRYPTO_ALG_TYPE_COMPRESS,
+	.type = CRYPTO_ALG_TYPE_ACOMPRESS,
 	.mask = CRYPTO_ALG_TYPE_MASK,
 };
 
@@ -833,8 +834,8 @@ void xfrm_probe_algs(void)
 	}
 
 	for (i = 0; i < calg_entries(); i++) {
-		status = crypto_has_comp(calg_list[i].name, 0,
-					 CRYPTO_ALG_ASYNC);
+		status = crypto_has_acomp(calg_list[i].name, 0,
+					  CRYPTO_ALG_ASYNC);
 		if (calg_list[i].available != status)
 			calg_list[i].available = status;
 	}
net/xfrm/xfrm_ipcomp.c +68 -39

206 lines not on this page · raw message

Signature · 1 line
2.39.2
#12 lore raw

Re: [RFC PATCH 04/21] net: ipcomp: Migrate to acomp API from deprecated comp API

Quoted 7 lines · Ard Biesheuvel

Migrate the IPcomp network compression code to the acomp API, in order to drop the dependency on the obsolete 'comp' API which is going away.

For the time being, this is a rather mechanical conversion replacing each comp TFM object with an acomp TFM/request object pair - this is necessary because, at this point, there is still a 1:1 relation between acomp tranforms and requests in the acomp-to-scomp adaptation layer, and

nit: tranforms -> transforms

...

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

#13 lore raw

RFC PATCH 5/21 ubifs: Pass worst-case buffer size to compression routines

In reply to Ard Biesheuvel

Currently, the ubifs code allocates a worst case buffer size to recompress a data node, but does not pass the size of that buffer to the compression code. This means that the compression code will never use the additional space, and might fail spuriously due to lack of space.

So let's multiply out_len by WORST_COMPR_FACTOR after allocating the buffer. Doing so is guaranteed not to overflow, given that the preceding kmalloc_array() call would have failed otherwise.

---
 fs/ubifs/journal.c | 2 ++
 1 file changed, 2 insertions(+)
fs/ubifs/journal.c +2 -0
index dc52ac0f4a345f30..4e5961878f336033 100644
@@ -1493,6 +1493,8 @@ static int truncate_data_node(const struct ubifs_info *c, const struct inode *in
 	if (!buf)
 		return -ENOMEM;
 
+	out_len *= WORST_COMPR_FACTOR;
+
 	dlen = le32_to_cpu(dn->ch.len) - UBIFS_DATA_NODE_SZ;
 	data_size = dn_size - UBIFS_DATA_NODE_SZ;
 	compr_type = le16_to_cpu(dn->compr_type);
Signature · 1 line
2.39.2
#14 lore raw

Re: [RFC PATCH 05/21] ubifs: Pass worst-case buffer size to compression routines

Quoted 19 lines · Ard Biesheuvel

Currently, the ubifs code allocates a worst case buffer size to recompress a data node, but does not pass the size of that buffer to the compression code. This means that the compression code will never use the additional space, and might fail spuriously due to lack of space.

So let's multiply out_len by WORST_COMPR_FACTOR after allocating the buffer. Doing so is guaranteed not to overflow, given that the preceding kmalloc_array() call would have failed otherwise.

---
 fs/ubifs/journal.c | 2 ++
 1 file changed, 2 insertions(+)
fs/ubifs/journal.c +0 -0
index dc52ac0f4a345f30..4e5961878f336033 100644

fs/ubifs/journal.c static int truncate_data_node(const struct ubifs_info *c, const struct inode *in · line 1500

Quoted 8 lines
fs/ubifs/journal.c +2 -0
@@ -1493,6 +1493,8 @@ static int truncate_data_node(const struct ubifs_info *c, const struct inode *in
 	if (!buf)
 		return -ENOMEM;
 
+	out_len *= WORST_COMPR_FACTOR;
+
 	dlen = le32_to_cpu(dn->ch.len) - UBIFS_DATA_NODE_SZ;
 	data_size = dn_size - UBIFS_DATA_NODE_SZ;
 	compr_type = le16_to_cpu(dn->compr_type);

This looks like another case where data that would be expanded by compression should just be stored uncompressed instead.

In fact, it seems that UBIFS does that already. ubifs_compress() has this:

/*
* If the data compressed only slightly, it is better to leave it
* uncompressed to improve read speed.
*/
if (in_len - *out_len < UBIFS_MIN_COMPRESS_DIFF)
goto no_compr;

So it's unclear why the WORST_COMPR_FACTOR thing is needed at all.

- Eric

#15 lore raw

Re: [RFC PATCH 05/21] ubifs: Pass worst-case buffer size to compression routines

Quoted 35 lines · Eric Biggers
Quoted 19 lines · Ard Biesheuvel

Currently, the ubifs code allocates a worst case buffer size to recompress a data node, but does not pass the size of that buffer to the compression code. This means that the compression code will never use the additional space, and might fail spuriously due to lack of space.

So let's multiply out_len by WORST_COMPR_FACTOR after allocating the buffer. Doing so is guaranteed not to overflow, given that the preceding kmalloc_array() call would have failed otherwise.

---
 fs/ubifs/journal.c | 2 ++
 1 file changed, 2 insertions(+)
fs/ubifs/journal.c +0 -0
index dc52ac0f4a345f30..4e5961878f336033 100644

fs/ubifs/journal.c static int truncate_data_node(const struct ubifs_info *c, const struct inode *in · line 1500

Quoted 8 lines
fs/ubifs/journal.c +2 -0
@@ -1493,6 +1493,8 @@ static int truncate_data_node(const struct ubifs_info *c, const struct inode *in
      if (!buf)
              return -ENOMEM;
 
+     out_len *= WORST_COMPR_FACTOR;
+
      dlen = le32_to_cpu(dn->ch.len) - UBIFS_DATA_NODE_SZ;
      data_size = dn_size - UBIFS_DATA_NODE_SZ;
      compr_type = le16_to_cpu(dn->compr_type);

This looks like another case where data that would be expanded by compression should just be stored uncompressed instead.

In fact, it seems that UBIFS does that already. ubifs_compress() has this:

Quoted 9 lines

/*
* If the data compressed only slightly, it is better to leave it
* uncompressed to improve read speed.
*/
if (in_len - *out_len < UBIFS_MIN_COMPRESS_DIFF)
goto no_compr;

So it's unclear why the WORST_COMPR_FACTOR thing is needed at all.

It is not. The buffer is used for decompression in the truncation path, so none of this logic even matters. Even if the subsequent recompression of the truncated data node could result in expansion beyond the uncompressed size of the original data (which seems impossible to me), increasing the size of this buffer would not help as it is the input buffer for the compression not the output buffer.

reviewer Reviewed-by
#16 lore raw

Re: [RFC PATCH 05/21] ubifs: Pass worst-case buffer size to compression routines

Quoted 6 lines · Ard Biesheuvel
Quoted 5 lines · Eric Biggers
Quoted 3 lines · Ard Biesheuvel

Currently, the ubifs code allocates a worst case buffer size to recompress a data node, but does not pass the size of that buffer to the compression code. This means that the compression code will never use

I think you mean the 'out_len' which describes the lengh of 'buf' is passed into ubifs_decompress, which effects the result of decompressor(eg. lz4 uses length to calculate the buffer end pos). So, we should pass the real lenghth of 'buf'.

Quoted 40 lines
Quoted 30 lines
Quoted 16 lines

the additional space, and might fail spuriously due to lack of space.

So let's multiply out_len by WORST_COMPR_FACTOR after allocating the buffer. Doing so is guaranteed not to overflow, given that the preceding kmalloc_array() call would have failed otherwise.

---
  fs/ubifs/journal.c | 2 ++
  1 file changed, 2 insertions(+)
fs/ubifs/journal.c +0 -0
index dc52ac0f4a345f30..4e5961878f336033 100644

fs/ubifs/journal.c static int truncate_data_node(const struct ubifs_info *c, const struct inode *in · line 1500

Quoted 8 lines
fs/ubifs/journal.c +2 -0
@@ -1493,6 +1493,8 @@ static int truncate_data_node(const struct ubifs_info *c, const struct inode *in
       if (!buf)
               return -ENOMEM;
 
+     out_len *= WORST_COMPR_FACTOR;
+
       dlen = le32_to_cpu(dn->ch.len) - UBIFS_DATA_NODE_SZ;
       data_size = dn_size - UBIFS_DATA_NODE_SZ;
       compr_type = le16_to_cpu(dn->compr_type);

This looks like another case where data that would be expanded by compression should just be stored uncompressed instead.

In fact, it seems that UBIFS does that already. ubifs_compress() has this:

Quoted 9 lines

/*
* If the data compressed only slightly, it is better to leave it
* uncompressed to improve read speed.
*/
if (in_len - *out_len < UBIFS_MIN_COMPRESS_DIFF)
goto no_compr;

So it's unclear why the WORST_COMPR_FACTOR thing is needed at all.

Quoted 8 lines

It is not. The buffer is used for decompression in the truncation path, so none of this logic even matters. Even if the subsequent recompression of the truncated data node could result in expansion beyond the uncompressed size of the original data (which seems impossible to me), increasing the size of this buffer would not help as it is the input buffer for the compression not the output buffer. .

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

#17 lore raw

Re: [RFC PATCH 05/21] ubifs: Pass worst-case buffer size to compression routines

Quoted 5 lines · Zhihao Cheng
Quoted 3 lines · Ard Biesheuvel
Quoted 2 lines · Eric Biggers

On Tue, Jul 18, 2023 at 02:58:31PM +0200, Ard Biesheuvel wrote:

Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Currently, the ubifs code allocates a worst case buffer size to recompress a data node, but does not pass the size of that buffer to the compression code. This means that the compression code will never use

I think you mean the 'out_len' which describes the lengh of 'buf' is passed into ubifs_decompress, which effects the result of decompressor(eg. lz4 uses length to calculate the buffer end pos). So, we should pass the real lenghth of 'buf'.

Yes, that is what I meant.

But Eric makes a good point, and looking a bit more closely, there is really no need for the multiplication here: we know the size of the decompressed data, so we don't need the additional space.

I intend to drop this patch, and replace it with the following:

----------------8<--------------

Currently, when truncating a data node, a decompression buffer is allocated that is twice the size of the data node's uncompressed size. However, the fact that this space is available is not communicated to the compression routines, as out_len itself is not updated.

The additional space is not needed even in the theoretical worst case where compression might lead to inadvertent expansion: first of all, increasing the size of the input buffer does not help mitigate that issue. And given the truncation of the data node and the fact that the original data compressed well enough to pass the UBIFS_MIN_COMPRESS_DIFF test, there is no way on this particular code path that compression could result in expansion beyond the original decompressed size, and so no mitigation is necessary to begin with.

So let's just drop WORST_COMPR_FACTOR here.

fs/ubifs/journal.c +0 -0
index dc52ac0f4a345f30..0b55cbfe0c30505e 100644
@@ -1489,7 +1489,7 @@ static int truncate_data_node(const struct

ubifs_info *c, const struct inode *in

        int err, dlen, compr_type, out_len, data_size;

        out_len = le32_to_cpu(dn->size);
-       buf = kmalloc_array(out_len, WORST_COMPR_FACTOR, GFP_NOFS);
+       buf = kmalloc(out_len, GFP_NOFS);
        if (!buf)
                return -ENOMEM;
reviewer Reviewed-by
#18 lore raw

Re: [RFC PATCH 05/21] ubifs: Pass worst-case buffer size to compression routines

Quoted 16 lines · Ard Biesheuvel
Quoted 5 lines · Zhihao Cheng
Quoted 3 lines · Ard Biesheuvel
Quoted 2 lines · Eric Biggers

On Tue, Jul 18, 2023 at 02:58:31PM +0200, Ard Biesheuvel wrote:

Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Currently, the ubifs code allocates a worst case buffer size to recompress a data node, but does not pass the size of that buffer to the compression code. This means that the compression code will never use

I think you mean the 'out_len' which describes the lengh of 'buf' is passed into ubifs_decompress, which effects the result of decompressor(eg. lz4 uses length to calculate the buffer end pos). So, we should pass the real lenghth of 'buf'.

Quoted 6 lines

Yes, that is what I meant.

But Eric makes a good point, and looking a bit more closely, there is really no need for the multiplication here: we know the size of the decompressed data, so we don't need the additional space.

Right, we get 'out_len' from 'dn->size' which is the length of uncompressed data. ubifs_compress makes sure the compressed length is smaller than original length.

Quoted 35 lines

I intend to drop this patch, and replace it with the following:

----------------8<--------------

Currently, when truncating a data node, a decompression buffer is allocated that is twice the size of the data node's uncompressed size. However, the fact that this space is available is not communicated to the compression routines, as out_len itself is not updated.

The additional space is not needed even in the theoretical worst case where compression might lead to inadvertent expansion: first of all, increasing the size of the input buffer does not help mitigate that issue. And given the truncation of the data node and the fact that the original data compressed well enough to pass the UBIFS_MIN_COMPRESS_DIFF test, there is no way on this particular code path that compression could result in expansion beyond the original decompressed size, and so no mitigation is necessary to begin with.

So let's just drop WORST_COMPR_FACTOR here.

fs/ubifs/journal.c +0 -0
index dc52ac0f4a345f30..0b55cbfe0c30505e 100644
@@ -1489,7 +1489,7 @@ static int truncate_data_node(const struct

ubifs_info *c, const struct inode *in

         int err, dlen, compr_type, out_len, data_size;

         out_len = le32_to_cpu(dn->size);
fs/ubifs/journal.c +1 -1
-       buf = kmalloc_array(out_len, WORST_COMPR_FACTOR, GFP_NOFS);
+       buf = kmalloc(out_len, GFP_NOFS);
         if (!buf)
                 return -ENOMEM;
Quoted 2 lines

.

This version looks better.

#19 lore raw

RFC PATCH 6/21 ubifs: Avoid allocating buffer space unnecessarily

In reply to Ard Biesheuvel

The recompression scratch buffer is only used when the data node is compressed, and there is no need to allocate it otherwise. So move the allocation into the branch of the if() that actually makes use of it.

---
 fs/ubifs/journal.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
fs/ubifs/journal.c +8 -8
index 4e5961878f336033..5ce618f82aed201b 100644
@@ -1485,16 +1485,9 @@ static int truncate_data_node(const struct ubifs_info *c, const struct inode *in
 			      unsigned int block, struct ubifs_data_node *dn,
 			      int *new_len, int dn_size)
 {
-	void *buf;
+	void *buf = NULL;
 	int err, dlen, compr_type, out_len, data_size;
 
-	out_len = le32_to_cpu(dn->size);
-	buf = kmalloc_array(out_len, WORST_COMPR_FACTOR, GFP_NOFS);
-	if (!buf)
-		return -ENOMEM;
-
-	out_len *= WORST_COMPR_FACTOR;
-
 	dlen = le32_to_cpu(dn->ch.len) - UBIFS_DATA_NODE_SZ;
 	data_size = dn_size - UBIFS_DATA_NODE_SZ;
 	compr_type = le16_to_cpu(dn->compr_type);
@@ -1508,6 +1501,13 @@ static int truncate_data_node(const struct ubifs_info *c, const struct inode *in
 	if (compr_type == UBIFS_COMPR_NONE) {
 		out_len = *new_len;
 	} else {
+		out_len = le32_to_cpu(dn->size);
+		buf = kmalloc_array(out_len, WORST_COMPR_FACTOR, GFP_NOFS);
+		if (!buf)
+			return -ENOMEM;
+
+		out_len *= WORST_COMPR_FACTOR;
+
 		err = ubifs_decompress(c, &dn->data, dlen, buf, &out_len, compr_type);
 		if (err)
 			goto out;
Signature · 1 line
2.39.2
#20 lore raw

RFC PATCH 7/21 ubifs: Migrate to acomp compression API

In reply to Ard Biesheuvel

UBIFS is one of the remaining users of the obsolete 'comp' compression API exposed by the crypto subsystem. Given that it operates strictly on contiguous buffers that are either entirely in lowmem or covered by a single page, the conversion to the acomp API is quite straight-forward.

Only synchronous acomp implementations are considered at the moment, and whether or not a future conversion to permit asynchronous ones too will be worth the effort remains to be seen.

---
 fs/ubifs/compress.c | 61 ++++++++++++++------
 fs/ubifs/file.c     | 46 ++++++++-------
 fs/ubifs/journal.c  | 19 ++++--
 fs/ubifs/ubifs.h    | 15 +++--
 4 files changed, 90 insertions(+), 51 deletions(-)
fs/ubifs/compress.c +42 -19
index 75461777c466b1c9..570919b218a0a8cc 100644
@@ -82,15 +82,15 @@ struct ubifs_compressor *ubifs_compressors[UBIFS_COMPR_TYPES_CNT];
 
 /**
  * ubifs_compress - compress data.
- * @in_buf: data to compress
+ * @in_sg: data to compress
  * @in_len: length of the data to compress
  * @out_buf: output buffer where compressed data should be stored
  * @out_len: output buffer length is returned here
  * @compr_type: type of compression to use on enter, actually used compression
  *              type on exit
  *
- * This function compresses input buffer @in_buf of length @in_len and stores
- * the result in the output buffer @out_buf and the resulting length in
+ * This function compresses input scatterlist @in_sg of length @in_len and
+ * stores the result in the output buffer @out_buf and the resulting length in
  * @out_len. If the input buffer does not compress, it is just copied to the
  * @out_buf. The same happens if @compr_type is %UBIFS_COMPR_NONE or if
  * compression error occurred.
@@ -98,11 +98,12 @@ struct ubifs_compressor *ubifs_compressors[UBIFS_COMPR_TYPES_CNT];
  * Note, if the input buffer was not compressed, it is copied to the output
  * buffer and %UBIFS_COMPR_NONE is returned in @compr_type.
  */
-void ubifs_compress(const struct ubifs_info *c, const void *in_buf,
+void ubifs_compress(const struct ubifs_info *c, struct scatterlist *in_sg,
 		    int in_len, void *out_buf, int *out_len, int *compr_type)
 {
 	int err;
 	struct ubifs_compressor *compr = ubifs_compressors[*compr_type];
+	struct scatterlist out_sg;
 
 	if (*compr_type == UBIFS_COMPR_NONE)
 		goto no_compr;
@@ -111,10 +112,13 @@ void ubifs_compress(const struct ubifs_info *c, const void *in_buf,
 	if (in_len < UBIFS_MIN_COMPR_LEN)
 		goto no_compr;
 
+	sg_init_one(&out_sg, out_buf, *out_len);
+
 	if (compr->comp_mutex)
 		mutex_lock(compr->comp_mutex);
-	err = crypto_comp_compress(compr->cc, in_buf, in_len, out_buf,
-				   (unsigned int *)out_len);
+	acomp_request_set_params(compr->req, in_sg, &out_sg, in_len, *out_len);
+	err = crypto_acomp_compress(compr->req);
+	*out_len = compr->req->dlen;
 	if (compr->comp_mutex)
 		mutex_unlock(compr->comp_mutex);
 	if (unlikely(err)) {
@@ -133,7 +137,7 @@ void ubifs_compress(const struct ubifs_info *c, const void *in_buf,
 	return;
 
 no_compr:
-	memcpy(out_buf, in_buf, in_len);
+	sg_copy_to_buffer(in_sg, 1, out_buf, in_len);
 	*out_len = in_len;
 	*compr_type = UBIFS_COMPR_NONE;
 }
@@ -142,19 +146,20 @@ void ubifs_compress(const struct ubifs_info *c, const void *in_buf,
  * ubifs_decompress - decompress data.
  * @in_buf: data to decompress
  * @in_len: length of the data to decompress
- * @out_buf: output buffer where decompressed data should
+ * @out_sg: output buffer where decompressed data should be stored
  * @out_len: output length is returned here
  * @compr_type: type of compression
  *
- * This function decompresses data from buffer @in_buf into buffer @out_buf.
+ * This function decompresses data from buffer @in_buf into scatterlist @out_sg.
  * The length of the uncompressed data is returned in @out_len. This functions
  * returns %0 on success or a negative error code on failure.
  */
-int ubifs_decompress(const struct ubifs_info *c, const void *in_buf,
-		     int in_len, void *out_buf, int *out_len, int compr_type)
+int ubifs_decompress(const struct ubifs_info *c, const void *in_buf, int in_len,
+		     struct scatterlist *out_sg, int *out_len, int compr_type)
 {
 	int err;
 	struct ubifs_compressor *compr;
+	struct scatterlist in_sg;
 
 	if (unlikely(compr_type < 0 || compr_type >= UBIFS_COMPR_TYPES_CNT)) {
 		ubifs_err(c, "invalid compression type %d", compr_type);
@@ -169,15 +174,18 @@ int ubifs_decompress(const struct ubifs_info *c, const void *in_buf,
 	}
 
 	if (compr_type == UBIFS_COMPR_NONE) {
-		memcpy(out_buf, in_buf, in_len);
+		sg_copy_from_buffer(out_sg, 1, in_buf, in_len);
 		*out_len = in_len;
 		return 0;
 	}
 
+	sg_init_one(&in_sg, in_buf, in_len);
+
 	if (compr->decomp_mutex)
 		mutex_lock(compr->decomp_mutex);
-	err = crypto_comp_decompress(compr->cc, in_buf, in_len, out_buf,
-				     (unsigned int *)out_len);
+	acomp_request_set_params(compr->req, &in_sg, out_sg, in_len, *out_len);
+	err = crypto_acomp_decompress(compr->req);
+	*out_len = compr->req->dlen;
 	if (compr->decomp_mutex)
 		mutex_unlock(compr->decomp_mutex);
 	if (err)
@@ -197,11 +205,24 @@ int ubifs_decompress(const struct ubifs_info *c, const void *in_buf,
 static int __init compr_init(struct ubifs_compressor *compr)
 {
 	if (compr->capi_name) {
-		compr->cc = crypto_alloc_comp(compr->capi_name, 0, 0);
+		long ret;
+
+		compr->cc = crypto_alloc_acomp(compr->capi_name, 0,
+					       CRYPTO_ALG_ASYNC);
 		if (IS_ERR(compr->cc)) {
+			ret = PTR_ERR(compr->cc);
+		} else {
+			compr->req = acomp_request_alloc(compr->cc);
+			if (!compr->req) {
+				crypto_free_acomp(compr->cc);
+				ret = -ENOMEM;
+			}
+		}
+
+		if (ret) {
 			pr_err("UBIFS error (pid %d): cannot initialize compressor %s, error %ld",
-			       current->pid, compr->name, PTR_ERR(compr->cc));
-			return PTR_ERR(compr->cc);
+			       current->pid, compr->name, ret);
+			return ret;
 		}
 	}
 
@@ -215,8 +236,10 @@ static int __init compr_init(struct ubifs_compressor *compr)
  */
 static void compr_exit(struct ubifs_compressor *compr)
 {
-	if (compr->capi_name)
-		crypto_free_comp(compr->cc);
+	if (compr->capi_name) {
+		acomp_request_free(compr->req);
+		crypto_free_acomp(compr->cc);
+	}
 }
 
 /**
fs/ubifs/file.c +25 -21

168 lines not on this page · raw message

fs/ubifs/journal.c +14 -5
index 5ce618f82aed201b..0d0f966c761e572a 100644
@@ -714,14 +714,16 @@ int ubifs_jnl_update(struct ubifs_info *c, const struct inode *dir,
  * @c: UBIFS file-system description object
  * @inode: inode the data node belongs to
  * @key: node key
- * @buf: buffer to write
+ * @page: struct page describing the page containing the source data
+ * @offset: offset in page to source data
  * @len: data length (must not exceed %UBIFS_BLOCK_SIZE)
  *
  * This function writes a data node to the journal. Returns %0 if the data node
  * was successfully written, and a negative error code in case of failure.
  */
 int ubifs_jnl_write_data(struct ubifs_info *c, const struct inode *inode,
-			 const union ubifs_key *key, const void *buf, int len)
+			 const union ubifs_key *key, struct page *page, int offset,
+			 int len)
 {
 	struct ubifs_data_node *data;
 	int err, lnum, offs, compr_type, out_len, compr_len, auth_len;
@@ -730,6 +732,7 @@ int ubifs_jnl_write_data(struct ubifs_info *c, const struct inode *inode,
 	struct ubifs_inode *ui = ubifs_inode(inode);
 	bool encrypted = IS_ENCRYPTED(inode);
 	u8 hash[UBIFS_HASH_ARR_SZ];
+	struct scatterlist sg;
 
 	dbg_jnlk(key, "ino %lu, blk %u, len %d, key ",
 		(unsigned long)key_inum(c, key), key_block(c, key), len);
@@ -765,7 +768,9 @@ int ubifs_jnl_write_data(struct ubifs_info *c, const struct inode *inode,
 		compr_type = ui->compr_type;
 
 	out_len = compr_len = dlen - UBIFS_DATA_NODE_SZ;
-	ubifs_compress(c, buf, len, &data->data, &compr_len, &compr_type);
+	sg_init_table(&sg, 1);
+	sg_set_page(&sg, page, len, offset);
+	ubifs_compress(c, &sg, len, &data->data, &compr_len, &compr_type);
 	ubifs_assert(c, compr_len <= UBIFS_BLOCK_SIZE);
 
 	if (encrypted) {
@@ -1487,6 +1492,7 @@ static int truncate_data_node(const struct ubifs_info *c, const struct inode *in
 {
 	void *buf = NULL;
 	int err, dlen, compr_type, out_len, data_size;
+	struct scatterlist sg_buf;
 
 	dlen = le32_to_cpu(dn->ch.len) - UBIFS_DATA_NODE_SZ;
 	data_size = dn_size - UBIFS_DATA_NODE_SZ;
@@ -1507,12 +1513,15 @@ static int truncate_data_node(const struct ubifs_info *c, const struct inode *in
 			return -ENOMEM;
 
 		out_len *= WORST_COMPR_FACTOR;
+		sg_init_one(&sg_buf, buf, out_len);
 
-		err = ubifs_decompress(c, &dn->data, dlen, buf, &out_len, compr_type);
+		err = ubifs_decompress(c, &dn->data, dlen, &sg_buf, &out_len,
+				       compr_type);
 		if (err)
 			goto out;
 
-		ubifs_compress(c, buf, *new_len, &dn->data, &out_len, &compr_type);
+		ubifs_compress(c, &sg_buf, *new_len, &dn->data, &out_len,
+			       &compr_type);
 	}
 
 	if (IS_ENCRYPTED(inode)) {
fs/ubifs/ubifs.h +9 -6
index 4c36044140e7eba9..2225de5b8ef50f71 100644
@@ -32,6 +32,7 @@
 #include <crypto/hash_info.h>
 #include <crypto/hash.h>
 #include <crypto/algapi.h>
+#include <crypto/acompress.h>
 
 #include <linux/fscrypt.h>
 
@@ -849,7 +850,8 @@ struct ubifs_node_range {
  */
 struct ubifs_compressor {
 	int compr_type;
-	struct crypto_comp *cc;
+	struct crypto_acomp *cc;
+	struct acomp_req *req;
 	struct mutex *comp_mutex;
 	struct mutex *decomp_mutex;
 	const char *name;
@@ -1801,7 +1803,8 @@ int ubifs_jnl_update(struct ubifs_info *c, const struct inode *dir,
 		     const struct fscrypt_name *nm, const struct inode *inode,
 		     int deletion, int xent);
 int ubifs_jnl_write_data(struct ubifs_info *c, const struct inode *inode,
-			 const union ubifs_key *key, const void *buf, int len);
+			 const union ubifs_key *key, struct page *page, int offset,
+			 int len);
 int ubifs_jnl_write_inode(struct ubifs_info *c, const struct inode *inode);
 int ubifs_jnl_delete_inode(struct ubifs_info *c, const struct inode *inode);
 int ubifs_jnl_xrename(struct ubifs_info *c, const struct inode *fst_dir,
@@ -2101,10 +2104,10 @@ long ubifs_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
 /* compressor.c */
 int __init ubifs_compressors_init(void);
 void ubifs_compressors_exit(void);
-void ubifs_compress(const struct ubifs_info *c, const void *in_buf, int in_len,
-		    void *out_buf, int *out_len, int *compr_type);
-int ubifs_decompress(const struct ubifs_info *c, const void *buf, int len,
-		     void *out, int *out_len, int compr_type);
+void ubifs_compress(const struct ubifs_info *c, struct scatterlist *in_sg,
+		    int in_len, void *out_buf, int *out_len, int *compr_type);
+int ubifs_decompress(const struct ubifs_info *c, const void *in_buf, int in_len,
+		     struct scatterlist *out_sg, int *out_len, int compr_type);
 
 /* sysfs.c */
 int ubifs_sysfs_init(void);
Signature · 1 line
2.39.2
#21 lore raw

Re: [RFC PATCH 07/21] ubifs: Migrate to acomp compression API

Quoted 10 lines · Ard Biesheuvel

UBIFS is one of the remaining users of the obsolete 'comp' compression API exposed by the crypto subsystem. Given that it operates strictly on contiguous buffers that are either entirely in lowmem or covered by a single page, the conversion to the acomp API is quite straight-forward.

Only synchronous acomp implementations are considered at the moment, and whether or not a future conversion to permit asynchronous ones too will be worth the effort remains to be seen.

...

Quoted 8 lines
fs/ubifs/compress.c +3 -1
@@ -197,11 +205,24 @@ int ubifs_decompress(const struct ubifs_info *c, const void *in_buf,
 static int __init compr_init(struct ubifs_compressor *compr)
 {
 	if (compr->capi_name) {
-		compr->cc = crypto_alloc_comp(compr->capi_name, 0, 0);
+		long ret;
+
+		compr->cc = crypto_alloc_acomp(compr->capi_name, 0,

fs/ubifs/compress.c int ubifs_decompress(const struct ubifs_info *c, const void *in_buf, · line 220

Quoted 10 lines
fs/ubifs/compress.c +9 -0
+					       CRYPTO_ALG_ASYNC);
 		if (IS_ERR(compr->cc)) {
+			ret = PTR_ERR(compr->cc);
+		} else {
+			compr->req = acomp_request_alloc(compr->cc);
+			if (!compr->req) {
+				crypto_free_acomp(compr->cc);
+				ret = -ENOMEM;
+			}
+		}

Hi Ard,

clang-16 W=1 and Smatch flag that ret may not always be initialised here.

Quoted 10 lines
fs/ubifs/compress.c +4 -2
+
+		if (ret) {
 			pr_err("UBIFS error (pid %d): cannot initialize compressor %s, error %ld",
-			       current->pid, compr->name, PTR_ERR(compr->cc));
-			return PTR_ERR(compr->cc);
+			       current->pid, compr->name, ret);
+			return ret;
 		}
 	}
 

...

Quoted 13 lines
fs/ubifs/ubifs.h +1 -0
index 4c36044140e7eba9..2225de5b8ef50f71 100644
@@ -32,6 +32,7 @@
 #include <crypto/hash_info.h>
 #include <crypto/hash.h>
 #include <crypto/algapi.h>
+#include <crypto/acompress.h>
 
 #include <linux/fscrypt.h>
 

fs/ubifs/ubifs.h struct ubifs_node_range { · line 854

Quoted 6 lines
fs/ubifs/ubifs.h +2 -1
@@ -849,7 +850,8 @@ struct ubifs_node_range {
  */
 struct ubifs_compressor {
 	int compr_type;
-	struct crypto_comp *cc;
+	struct crypto_acomp *cc;
+	struct acomp_req *req;

Please consider adding @req to the kernel doc for this structure.

Quoted 3 lines
fs/ubifs/ubifs.h +0 -0
 	struct mutex *comp_mutex;
 	struct mutex *decomp_mutex;
 	const char *name;

...

#22 lore raw

RFC PATCH 8/21 zram: Migrate to acomp compression API

In reply to Ard Biesheuvel

Switch from the deprecated 'comp' to the more recent 'acomp' API.

This involves using scatterlists and request objects to describe the in- and output buffers, all of which happen to be contiguous in memory, and reside either entirely in lowmem, or inside a single highmem page. This makes the conversion quite straight-forward, and easy to back by either a software or a hardware implementation.

---
 drivers/block/zram/zcomp.c    | 67 +++++++++++++++-----
 drivers/block/zram/zcomp.h    |  7 +-
 drivers/block/zram/zram_drv.c | 12 +---
 3 files changed, 57 insertions(+), 29 deletions(-)
drivers/block/zram/zcomp.c +50 -17
index 55af4efd79835666..12bdd288a153c455 100644
@@ -11,6 +11,9 @@
 #include <linux/sched.h>
 #include <linux/cpu.h>
 #include <linux/crypto.h>
+#include <linux/highmem.h>
+#include <linux/scatterlist.h>
+#include <crypto/acompress.h>
 
 #include "zcomp.h"
 
@@ -35,26 +38,32 @@ static const char * const backends[] = {
 
 static void zcomp_strm_free(struct zcomp_strm *zstrm)
 {
+	if (zstrm->req)
+		acomp_request_free(zstrm->req);
 	if (!IS_ERR_OR_NULL(zstrm->tfm))
-		crypto_free_comp(zstrm->tfm);
+		crypto_free_acomp(zstrm->tfm);
 	free_pages((unsigned long)zstrm->buffer, 1);
+	zstrm->req = NULL;
 	zstrm->tfm = NULL;
 	zstrm->buffer = NULL;
 }
 
 /*
- * Initialize zcomp_strm structure with ->tfm initialized by backend, and
- * ->buffer. Return a negative value on error.
+ * Initialize zcomp_strm structure with ->tfm and ->req initialized by
+ * backend, and ->buffer. Return a negative value on error.
  */
 static int zcomp_strm_init(struct zcomp_strm *zstrm, struct zcomp *comp)
 {
-	zstrm->tfm = crypto_alloc_comp(comp->name, 0, 0);
+	zstrm->tfm = crypto_alloc_acomp(comp->name, 0, CRYPTO_ALG_ASYNC);
+	if (!IS_ERR_OR_NULL(zstrm->tfm))
+		zstrm->req = acomp_request_alloc(zstrm->tfm);
+
 	/*
 	 * allocate 2 pages. 1 for compressed data, plus 1 extra for the
 	 * case when compressed size is larger than the original one
 	 */
 	zstrm->buffer = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, 1);
-	if (IS_ERR_OR_NULL(zstrm->tfm) || !zstrm->buffer) {
+	if (IS_ERR_OR_NULL(zstrm->tfm) || !zstrm->req || !zstrm->buffer) {
 		zcomp_strm_free(zstrm);
 		return -ENOMEM;
 	}
@@ -70,7 +79,7 @@ bool zcomp_available_algorithm(const char *comp)
 	 * This also means that we permit zcomp initialisation
 	 * with any compressing algorithm known to crypto api.
 	 */
-	return crypto_has_comp(comp, 0, 0) == 1;
+	return crypto_has_acomp(comp, 0, CRYPTO_ALG_ASYNC);
 }
 
 /* show available compressors */
@@ -95,7 +104,7 @@ ssize_t zcomp_available_show(const char *comp, char *buf)
 	 * Out-of-tree module known to crypto api or a missing
 	 * entry in `backends'.
 	 */
-	if (!known_algorithm && crypto_has_comp(comp, 0, 0) == 1)
+	if (!known_algorithm && crypto_has_acomp(comp, 0, CRYPTO_ALG_ASYNC))
 		sz += scnprintf(buf + sz, PAGE_SIZE - sz - 2,
 				"[%s] ", comp);
 
@@ -115,8 +124,14 @@ void zcomp_stream_put(struct zcomp *comp)
 }
 
 int zcomp_compress(struct zcomp_strm *zstrm,
-		const void *src, unsigned int *dst_len)
+		   struct page *src, unsigned int *dst_len)
 {
+	struct scatterlist sg_src, sg_dst;
+	int ret;
+
+	sg_init_table(&sg_src, 1);
+	sg_set_page(&sg_src, src, PAGE_SIZE, 0);
+
 	/*
 	 * Our dst memory (zstrm->buffer) is always `2 * PAGE_SIZE' sized
 	 * because sometimes we can endup having a bigger compressed data
@@ -131,21 +146,39 @@ int zcomp_compress(struct zcomp_strm *zstrm,
 	 * the dst buffer, zram_drv will take care of the fact that
 	 * compressed buffer is too big.
 	 */
-	*dst_len = PAGE_SIZE * 2;
+	sg_init_one(&sg_dst, zstrm->buffer, PAGE_SIZE * 2);
 
-	return crypto_comp_compress(zstrm->tfm,
-			src, PAGE_SIZE,
-			zstrm->buffer, dst_len);
+	acomp_request_set_params(zstrm->req, &sg_src, &sg_dst, PAGE_SIZE,
+				 PAGE_SIZE * 2);
+
+	ret = crypto_acomp_compress(zstrm->req);
+	if (ret)
+		return ret;
+
+	*dst_len = zstrm->req->dlen;
+	return 0;
 }
 
 int zcomp_decompress(struct zcomp_strm *zstrm,
-		const void *src, unsigned int src_len, void *dst)
+		     const void *src, unsigned int src_len, struct page *dst)
 {
-	unsigned int dst_len = PAGE_SIZE;
+	struct scatterlist sg_src, sg_dst;
 
-	return crypto_comp_decompress(zstrm->tfm,
-			src, src_len,
-			dst, &dst_len);
+	if (is_kmap_addr(src)) {
+		sg_init_table(&sg_src, 1);
+		sg_set_page(&sg_src, kmap_to_page((void *)src), src_len,
+			    offset_in_page(src));
+	} else {
+		sg_init_one(&sg_src, src, src_len);
+	}
+
+	sg_init_table(&sg_dst, 1);
+	sg_set_page(&sg_dst, dst, PAGE_SIZE, 0);
+
+	acomp_request_set_params(zstrm->req, &sg_src, &sg_dst, src_len,
+				 PAGE_SIZE);
+
+	return crypto_acomp_decompress(zstrm->req);
 }
 
 int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node)
drivers/block/zram/zcomp.h +4 -3
index cdefdef93da8c00d..32b9c4ae2d6dd9bf 100644
@@ -12,7 +12,8 @@ struct zcomp_strm {
 	local_lock_t lock;
 	/* compression/decompression buffer */
 	void *buffer;
-	struct crypto_comp *tfm;
+	struct crypto_acomp *tfm;
+	struct acomp_req *req;
 };
 
 /* dynamic per-device compression frontend */
@@ -34,10 +35,10 @@ struct zcomp_strm *zcomp_stream_get(struct zcomp *comp);
 void zcomp_stream_put(struct zcomp *comp);
 
 int zcomp_compress(struct zcomp_strm *zstrm,
-		const void *src, unsigned int *dst_len);
+		   struct page *src, unsigned int *dst_len);
 
 int zcomp_decompress(struct zcomp_strm *zstrm,
-		const void *src, unsigned int src_len, void *dst);
+		     const void *src, unsigned int src_len, struct page *dst);
 
 bool zcomp_set_max_streams(struct zcomp *comp, int num_strm);
 #endif /* _ZCOMP_H_ */
drivers/block/zram/zram_drv.c +3 -9
index 5676e6dd5b1672a8..ac24f5d955e3c16d 100644
@@ -1345,9 +1345,7 @@ static int zram_read_from_zspool(struct zram *zram, struct page *page,
 		kunmap_atomic(dst);
 		ret = 0;
 	} else {
-		dst = kmap_atomic(page);
-		ret = zcomp_decompress(zstrm, src, size, dst);
-		kunmap_atomic(dst);
+		ret = zcomp_decompress(zstrm, src, size, page);
 		zcomp_stream_put(zram->comps[prio]);
 	}
 	zs_unmap_object(zram->mem_pool, handle);
@@ -1432,9 +1430,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index)
 
 compress_again:
 	zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]);
-	src = kmap_atomic(page);
-	ret = zcomp_compress(zstrm, src, &comp_len);
-	kunmap_atomic(src);
+	ret = zcomp_compress(zstrm, page, &comp_len);
 
 	if (unlikely(ret)) {
 		zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]);
@@ -1618,9 +1614,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
 
 		num_recomps++;
 		zstrm = zcomp_stream_get(zram->comps[prio]);
-		src = kmap_atomic(page);
-		ret = zcomp_compress(zstrm, src, &comp_len_new);
-		kunmap_atomic(src);
+		ret = zcomp_compress(zstrm, page, &comp_len_new);
 
 		if (ret) {
 			zcomp_stream_put(zram->comps[prio]);
Signature · 1 line
2.39.2
#23 lore raw

Re: [RFC PATCH 08/21] zram: Migrate to acomp compression API

kernel test robot noticed the following build warnings:

Hi Ard,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:

[auto build test WARNING on herbert-cryptodev-2.6/master]
[also build test WARNING on herbert-crypto-2.6/master linus/master v6.5-rc2 next-20230718] [cannot apply to rw-ubifs/next rw-ubifs/fixes]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ard-Biesheuvel/crypto-scomp-Revert-add-support-for-deflate-rfc1950-zlib/20230718-210600
base:   https://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git master
patch link:    https://lore.kernel.org/r/20230718125847.3869700-9-ardb%40kernel.org
patch subject: [RFC PATCH 08/21] zram: Migrate to acomp compression API
config: x86_64-buildonly-randconfig-r001-20230718 (https://download.01.org/0day-ci/archive/20230719/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230719/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

drivers/block/zram/zram_drv.c: In function 'zram_recompress':

>> drivers/block/zram/zram_drv.c:1581:15: warning: unused variable 'src' [-Wunused-variable]
    1581 |         void *src, *dst;
         |               ^~~

vim +/src +1581 drivers/block/zram/zram_drv.c

a0b81ae7a4ff8a7 Christoph Hellwig  2023-04-11  1561  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1562  #ifdef CONFIG_ZRAM_MULTI_COMP
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1563  /*
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1564   * This function will decompress (unless it's ZRAM_HUGE) the page and then
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1565   * attempt to compress it using provided compression algorithm priority
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1566   * (which is potentially more effective).
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1567   *
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1568   * Corresponding ZRAM slot should be locked.
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1569   */
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1570  static int zram_recompress(struct zram *zram, u32 index, struct page *page,
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1571  			   u32 threshold, u32 prio, u32 prio_max)
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1572  {
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1573  	struct zcomp_strm *zstrm = NULL;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1574  	unsigned long handle_old;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1575  	unsigned long handle_new;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1576  	unsigned int comp_len_old;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1577  	unsigned int comp_len_new;
7c2af309abd24ff Alexey Romanov     2022-11-09  1578  	unsigned int class_index_old;
7c2af309abd24ff Alexey Romanov     2022-11-09  1579  	unsigned int class_index_new;
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1580  	u32 num_recomps = 0;
84b33bf7888975d Sergey Senozhatsky 2022-11-09 @1581  	void *src, *dst;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1582  	int ret;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1583  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1584  	handle_old = zram_get_handle(zram, index);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1585  	if (!handle_old)
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1586  		return -EINVAL;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1587  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1588  	comp_len_old = zram_get_obj_size(zram, index);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1589  	/*
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1590  	 * Do not recompress objects that are already "small enough".
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1591  	 */
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1592  	if (comp_len_old < threshold)
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1593  		return 0;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1594  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1595  	ret = zram_read_from_zspool(zram, page, index);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1596  	if (ret)
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1597  		return ret;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1598  
7c2af309abd24ff Alexey Romanov     2022-11-09  1599  	class_index_old = zs_lookup_class_index(zram->mem_pool, comp_len_old);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1600  	/*
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1601  	 * Iterate the secondary comp algorithms list (in order of priority)
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1602  	 * and try to recompress the page.
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1603  	 */
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1604  	for (; prio < prio_max; prio++) {
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1605  		if (!zram->comps[prio])
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1606  			continue;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1607  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1608  		/*
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1609  		 * Skip if the object is already re-compressed with a higher
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1610  		 * priority algorithm (or same algorithm).
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1611  		 */
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1612  		if (prio <= zram_get_priority(zram, index))
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1613  			continue;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1614  
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1615  		num_recomps++;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1616  		zstrm = zcomp_stream_get(zram->comps[prio]);
29b6967174757a8 Ard Biesheuvel     2023-07-18  1617  		ret = zcomp_compress(zstrm, page, &comp_len_new);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1618  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1619  		if (ret) {
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1620  			zcomp_stream_put(zram->comps[prio]);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1621  			return ret;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1622  		}
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1623  
7c2af309abd24ff Alexey Romanov     2022-11-09  1624  		class_index_new = zs_lookup_class_index(zram->mem_pool,
7c2af309abd24ff Alexey Romanov     2022-11-09  1625  							comp_len_new);
7c2af309abd24ff Alexey Romanov     2022-11-09  1626  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1627  		/* Continue until we make progress */
4942cf6ad07c487 Sergey Senozhatsky 2022-11-09  1628  		if (class_index_new >= class_index_old ||
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1629  		    (threshold && comp_len_new >= threshold)) {
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1630  			zcomp_stream_put(zram->comps[prio]);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1631  			continue;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1632  		}
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1633  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1634  		/* Recompression was successful so break out */
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1635  		break;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1636  	}
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1637  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1638  	/*
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1639  	 * We did not try to recompress, e.g. when we have only one
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1640  	 * secondary algorithm and the page is already recompressed
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1641  	 * using that algorithm
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1642  	 */
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1643  	if (!zstrm)
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1644  		return 0;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1645  
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1646  	if (class_index_new >= class_index_old) {
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1647  		/*
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1648  		 * Secondary algorithms failed to re-compress the page
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1649  		 * in a way that would save memory, mark the object as
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1650  		 * incompressible so that we will not try to compress
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1651  		 * it again.
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1652  		 *
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1653  		 * We need to make sure that all secondary algorithms have
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1654  		 * failed, so we test if the number of recompressions matches
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1655  		 * the number of active secondary algorithms.
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1656  		 */
a55cf9648d3de48 Sergey Senozhatsky 2022-11-09  1657  		if (num_recomps == zram->num_active_comps - 1)
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1658  			zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1659  		return 0;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1660  	}
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1661  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1662  	/* Successful recompression but above threshold */
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1663  	if (threshold && comp_len_new >= threshold)
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1664  		return 0;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1665  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1666  	/*
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1667  	 * No direct reclaim (slow path) for handle allocation and no
6aa4b839e7a481c Christoph Hellwig  2023-04-11  1668  	 * re-compression attempt (unlike in zram_write_bvec()) since
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1669  	 * we already have stored that object in zsmalloc. If we cannot
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1670  	 * alloc memory for recompressed object then we bail out and
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1671  	 * simply keep the old (existing) object in zsmalloc.
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1672  	 */
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1673  	handle_new = zs_malloc(zram->mem_pool, comp_len_new,
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1674  			       __GFP_KSWAPD_RECLAIM |
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1675  			       __GFP_NOWARN |
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1676  			       __GFP_HIGHMEM |
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1677  			       __GFP_MOVABLE);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1678  	if (IS_ERR_VALUE(handle_new)) {
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1679  		zcomp_stream_put(zram->comps[prio]);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1680  		return PTR_ERR((void *)handle_new);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1681  	}
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1682  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1683  	dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1684  	memcpy(dst, zstrm->buffer, comp_len_new);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1685  	zcomp_stream_put(zram->comps[prio]);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1686  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1687  	zs_unmap_object(zram->mem_pool, handle_new);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1688  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1689  	zram_free_page(zram, index);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1690  	zram_set_handle(zram, index, handle_new);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1691  	zram_set_obj_size(zram, index, comp_len_new);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1692  	zram_set_priority(zram, index, prio);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1693  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1694  	atomic64_add(comp_len_new, &zram->stats.compr_data_size);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1695  	atomic64_inc(&zram->stats.pages_stored);
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1696  
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1697  	return 0;
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1698  }
84b33bf7888975d Sergey Senozhatsky 2022-11-09  1699  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#24 lore raw

Re: [RFC PATCH 08/21] zram: Migrate to acomp compression API

In reply to Ard Biesheuvel

On Tue, Jul 18, 2023 at 02:58:34PM +0200, Ard Biesheuvel wrote:

...

drivers/block/zram/zram_drv.c static int zram_recompress(struct zram *zram, u32 index, struct page *page, · line 1617

Quoted 8 lines
drivers/block/zram/zram_drv.c +1 -3
@@ -1618,9 +1614,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
 
 		num_recomps++;
 		zstrm = zcomp_stream_get(zram->comps[prio]);
-		src = kmap_atomic(page);
-		ret = zcomp_compress(zstrm, src, &comp_len_new);
-		kunmap_atomic(src);
+		ret = zcomp_compress(zstrm, page, &comp_len_new);

Hi Ard,

src appears to be unused in this function with this change.

drivers/block/zram/zram_drv.c static int zram_recompress(struct zram *zram, u32 index, struct page *page, · line 1620

Quoted 3 lines
drivers/block/zram/zram_drv.c +0 -0
 		if (ret) {
 			zcomp_stream_put(zram->comps[prio]);

...

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

#25 lore raw

RFC PATCH 9/21 crypto: nx - Migrate to scomp API

In reply to Ard Biesheuvel

The only remaining user of 842 compression has been migrated to the acomp compression API, and so the NX hardware driver has to follow suit, given that no users of the obsolete 'comp' API remain, and it is going to be removed.

So migrate the NX driver code to scomp. These will be wrapped and exposed as acomp implementation via the crypto subsystem's acomp-to-scomp adaptation layer.

---
 drivers/crypto/nx/nx-842.c            | 34 ++++++++++++--------
 drivers/crypto/nx/nx-842.h            | 14 ++++----
 drivers/crypto/nx/nx-common-powernv.c | 30 ++++++++---------
 drivers/crypto/nx/nx-common-pseries.c | 32 +++++++++---------
 4 files changed, 57 insertions(+), 53 deletions(-)
drivers/crypto/nx/nx-842.c +20 -14
index 2ab90ec10e61ebe8..331b9cdf85e27044 100644
@@ -101,9 +101,14 @@ static int update_param(struct nx842_crypto_param *p,
 	return 0;
 }
 
-int nx842_crypto_init(struct crypto_tfm *tfm, struct nx842_driver *driver)
+void *nx842_crypto_alloc_ctx(struct crypto_scomp *tfm,
+			     struct nx842_driver *driver)
 {
-	struct nx842_crypto_ctx *ctx = crypto_tfm_ctx(tfm);
+	struct nx842_crypto_ctx *ctx;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return ERR_PTR(-ENOMEM);
 
 	spin_lock_init(&ctx->lock);
 	ctx->driver = driver;
@@ -114,22 +119,23 @@ int nx842_crypto_init(struct crypto_tfm *tfm, struct nx842_driver *driver)
 		kfree(ctx->wmem);
 		free_page((unsigned long)ctx->sbounce);
 		free_page((unsigned long)ctx->dbounce);
-		return -ENOMEM;
+		kfree(ctx);
+		return ERR_PTR(-ENOMEM);
 	}
 
-	return 0;
+	return ctx;
 }
-EXPORT_SYMBOL_GPL(nx842_crypto_init);
+EXPORT_SYMBOL_GPL(nx842_crypto_alloc_ctx);
 
-void nx842_crypto_exit(struct crypto_tfm *tfm)
+void nx842_crypto_free_ctx(struct crypto_scomp *tfm, void *p)
 {
-	struct nx842_crypto_ctx *ctx = crypto_tfm_ctx(tfm);
+	struct nx842_crypto_ctx *ctx = p;
 
 	kfree(ctx->wmem);
 	free_page((unsigned long)ctx->sbounce);
 	free_page((unsigned long)ctx->dbounce);
 }
-EXPORT_SYMBOL_GPL(nx842_crypto_exit);
+EXPORT_SYMBOL_GPL(nx842_crypto_free_ctx);
 
 static void check_constraints(struct nx842_constraints *c)
 {
@@ -246,11 +252,11 @@ static int compress(struct nx842_crypto_ctx *ctx,
 	return update_param(p, slen, dskip + dlen);
 }
 
-int nx842_crypto_compress(struct crypto_tfm *tfm,
+int nx842_crypto_compress(struct crypto_scomp *tfm,
 			  const u8 *src, unsigned int slen,
-			  u8 *dst, unsigned int *dlen)
+			  u8 *dst, unsigned int *dlen, void *pctx)
 {
-	struct nx842_crypto_ctx *ctx = crypto_tfm_ctx(tfm);
+	struct nx842_crypto_ctx *ctx = pctx;
 	struct nx842_crypto_header *hdr = &ctx->header;
 	struct nx842_crypto_param p;
 	struct nx842_constraints c = *ctx->driver->constraints;
@@ -429,11 +435,11 @@ static int decompress(struct nx842_crypto_ctx *ctx,
 	return update_param(p, slen + padding, dlen);
 }
 
-int nx842_crypto_decompress(struct crypto_tfm *tfm,
+int nx842_crypto_decompress(struct crypto_scomp *tfm,
 			    const u8 *src, unsigned int slen,
-			    u8 *dst, unsigned int *dlen)
+			    u8 *dst, unsigned int *dlen, void *pctx)
 {
-	struct nx842_crypto_ctx *ctx = crypto_tfm_ctx(tfm);
+	struct nx842_crypto_ctx *ctx = pctx;
 	struct nx842_crypto_header *hdr;
 	struct nx842_crypto_param p;
 	struct nx842_constraints c = *ctx->driver->constraints;
drivers/crypto/nx/nx-842.h +8 -6
index 7590bfb24d79bf42..de9dc8df62ed9dcb 100644
@@ -12,6 +12,7 @@
 #include <linux/io.h>
 #include <linux/mm.h>
 #include <linux/ratelimit.h>
+#include <crypto/internal/scompress.h>
 
 /* Restrictions on Data Descriptor List (DDL) and Entry (DDE) buffers
  *
@@ -177,13 +178,14 @@ struct nx842_crypto_ctx {
 	struct nx842_driver *driver;
 };
 
-int nx842_crypto_init(struct crypto_tfm *tfm, struct nx842_driver *driver);
-void nx842_crypto_exit(struct crypto_tfm *tfm);
-int nx842_crypto_compress(struct crypto_tfm *tfm,
+void *nx842_crypto_alloc_ctx(struct crypto_scomp *tfm,
+			     struct nx842_driver *driver);
+void nx842_crypto_free_ctx(struct crypto_scomp *tfm, void *ctx);
+int nx842_crypto_compress(struct crypto_scomp *tfm,
 			  const u8 *src, unsigned int slen,
-			  u8 *dst, unsigned int *dlen);
-int nx842_crypto_decompress(struct crypto_tfm *tfm,
+			  u8 *dst, unsigned int *dlen, void *ctx);
+int nx842_crypto_decompress(struct crypto_scomp *tfm,
 			    const u8 *src, unsigned int slen,
-			    u8 *dst, unsigned int *dlen);
+			    u8 *dst, unsigned int *dlen, void *ctx);
 
 #endif /* __NX_842_H__ */
drivers/crypto/nx/nx-common-powernv.c +14 -16
index 8c859872c1839eca..80023686f3e21b72 100644
@@ -1031,23 +1031,21 @@ static struct nx842_driver nx842_powernv_driver = {
 	.decompress =	nx842_powernv_decompress,
 };
 
-static int nx842_powernv_crypto_init(struct crypto_tfm *tfm)
+static void *nx842_powernv_crypto_alloc_ctx(struct crypto_scomp *tfm)
 {
-	return nx842_crypto_init(tfm, &nx842_powernv_driver);
+	return nx842_crypto_alloc_ctx(tfm, &nx842_powernv_driver);
 }
 
-static struct crypto_alg nx842_powernv_alg = {
-	.cra_name		= "842",
-	.cra_driver_name	= "842-nx",
-	.cra_priority		= 300,
-	.cra_flags		= CRYPTO_ALG_TYPE_COMPRESS,
-	.cra_ctxsize		= sizeof(struct nx842_crypto_ctx),
-	.cra_module		= THIS_MODULE,
-	.cra_init		= nx842_powernv_crypto_init,
-	.cra_exit		= nx842_crypto_exit,
-	.cra_u			= { .compress = {
-	.coa_compress		= nx842_crypto_compress,
-	.coa_decompress		= nx842_crypto_decompress } }
+static struct scomp_alg nx842_powernv_alg = {
+	.base.cra_name		= "842",
+	.base.cra_driver_name	= "842-nx",
+	.base.cra_priority	= 300,
+	.base.cra_module	= THIS_MODULE,
+
+	.alloc_ctx		= nx842_powernv_crypto_alloc_ctx,
+	.free_ctx		= nx842_crypto_free_ctx,
+	.compress		= nx842_crypto_compress,
+	.decompress		= nx842_crypto_decompress,
 };
 
 static __init int nx_compress_powernv_init(void)
@@ -1107,7 +1105,7 @@ static __init int nx_compress_powernv_init(void)
 		nx842_powernv_exec = nx842_exec_vas;
 	}
 
-	ret = crypto_register_alg(&nx842_powernv_alg);
+	ret = crypto_register_scomp(&nx842_powernv_alg);
 	if (ret) {
 		nx_delete_coprocs();
 		return ret;
@@ -1128,7 +1126,7 @@ static void __exit nx_compress_powernv_exit(void)
 	if (!nx842_ct)
 		vas_unregister_api_powernv();
 
-	crypto_unregister_alg(&nx842_powernv_alg);
+	crypto_unregister_scomp(&nx842_powernv_alg);
 
 	nx_delete_coprocs();
 }
drivers/crypto/nx/nx-common-pseries.c +15 -17
index 35f2d0d8507ed774..6232901adc495ed3 100644
@@ -1008,23 +1008,21 @@ static struct nx842_driver nx842_pseries_driver = {
 	.decompress =	nx842_pseries_decompress,
 };
 
-static int nx842_pseries_crypto_init(struct crypto_tfm *tfm)
+static void *nx842_pseries_crypto_alloc_ctx(struct crypto_scomp *tfm)
 {
-	return nx842_crypto_init(tfm, &nx842_pseries_driver);
+	return nx842_crypto_alloc_ctx(tfm, &nx842_pseries_driver);
 }
 
-static struct crypto_alg nx842_pseries_alg = {
-	.cra_name		= "842",
-	.cra_driver_name	= "842-nx",
-	.cra_priority		= 300,
-	.cra_flags		= CRYPTO_ALG_TYPE_COMPRESS,
-	.cra_ctxsize		= sizeof(struct nx842_crypto_ctx),
-	.cra_module		= THIS_MODULE,
-	.cra_init		= nx842_pseries_crypto_init,
-	.cra_exit		= nx842_crypto_exit,
-	.cra_u			= { .compress = {
-	.coa_compress		= nx842_crypto_compress,
-	.coa_decompress		= nx842_crypto_decompress } }
+static struct scomp_alg nx842_pseries_alg = {
+	.base.cra_name		= "842",
+	.base.cra_driver_name	= "842-nx",
+	.base.cra_priority	= 300,
+	.base.cra_module	= THIS_MODULE,
+
+	.alloc_ctx		= nx842_pseries_crypto_alloc_ctx,
+	.free_ctx		= nx842_crypto_free_ctx,
+	.compress		= nx842_crypto_compress,
+	.decompress		= nx842_crypto_decompress,
 };
 
 static int nx842_probe(struct vio_dev *viodev,
@@ -1072,7 +1070,7 @@ static int nx842_probe(struct vio_dev *viodev,
 	if (ret)
 		goto error;
 
-	ret = crypto_register_alg(&nx842_pseries_alg);
+	ret = crypto_register_scomp(&nx842_pseries_alg);
 	if (ret) {
 		dev_err(&viodev->dev, "could not register comp alg: %d\n", ret);
 		goto error;
@@ -1120,7 +1118,7 @@ static void nx842_remove(struct vio_dev *viodev)
 	if (caps_feat)
 		sysfs_remove_group(&viodev->dev.kobj, &nxcop_caps_attr_group);
 
-	crypto_unregister_alg(&nx842_pseries_alg);
+	crypto_unregister_scomp(&nx842_pseries_alg);
 
 	spin_lock_irqsave(&devdata_mutex, flags);
 	old_devdata = rcu_dereference_check(devdata,
@@ -1255,7 +1253,7 @@ static void __exit nx842_pseries_exit(void)
 
 	vas_unregister_api_pseries();
 
-	crypto_unregister_alg(&nx842_pseries_alg);
+	crypto_unregister_scomp(&nx842_pseries_alg);
 
 	spin_lock_irqsave(&devdata_mutex, flags);
 	old_devdata = rcu_dereference_check(devdata,
Signature · 1 line
2.39.2
#26 lore raw

RFC PATCH 10/21 crypto: 842 - drop obsolete 'comp' implementation

In reply to Ard Biesheuvel

The 'comp' API is obsolete and will be removed, so remove this comp implementation.

---
 crypto/842.c | 63 +-------------------
 1 file changed, 1 insertion(+), 62 deletions(-)
crypto/842.c +1 -62
index e59e54d769609ba6..5001d88cf727f74e 100644
@@ -39,38 +39,11 @@ static void *crypto842_alloc_ctx(struct crypto_scomp *tfm)
 	return ctx;
 }
 
-static int crypto842_init(struct crypto_tfm *tfm)
-{
-	struct crypto842_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	ctx->wmem = crypto842_alloc_ctx(NULL);
-	if (IS_ERR(ctx->wmem))
-		return -ENOMEM;
-
-	return 0;
-}
-
 static void crypto842_free_ctx(struct crypto_scomp *tfm, void *ctx)
 {
 	kfree(ctx);
 }
 
-static void crypto842_exit(struct crypto_tfm *tfm)
-{
-	struct crypto842_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	crypto842_free_ctx(NULL, ctx->wmem);
-}
-
-static int crypto842_compress(struct crypto_tfm *tfm,
-			      const u8 *src, unsigned int slen,
-			      u8 *dst, unsigned int *dlen)
-{
-	struct crypto842_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	return sw842_compress(src, slen, dst, dlen, ctx->wmem);
-}
-
 static int crypto842_scompress(struct crypto_scomp *tfm,
 			       const u8 *src, unsigned int slen,
 			       u8 *dst, unsigned int *dlen, void *ctx)
@@ -78,13 +51,6 @@ static int crypto842_scompress(struct crypto_scomp *tfm,
 	return sw842_compress(src, slen, dst, dlen, ctx);
 }
 
-static int crypto842_decompress(struct crypto_tfm *tfm,
-				const u8 *src, unsigned int slen,
-				u8 *dst, unsigned int *dlen)
-{
-	return sw842_decompress(src, slen, dst, dlen);
-}
-
 static int crypto842_sdecompress(struct crypto_scomp *tfm,
 				 const u8 *src, unsigned int slen,
 				 u8 *dst, unsigned int *dlen, void *ctx)
@@ -92,20 +58,6 @@ static int crypto842_sdecompress(struct crypto_scomp *tfm,
 	return sw842_decompress(src, slen, dst, dlen);
 }
 
-static struct crypto_alg alg = {
-	.cra_name		= "842",
-	.cra_driver_name	= "842-generic",
-	.cra_priority		= 100,
-	.cra_flags		= CRYPTO_ALG_TYPE_COMPRESS,
-	.cra_ctxsize		= sizeof(struct crypto842_ctx),
-	.cra_module		= THIS_MODULE,
-	.cra_init		= crypto842_init,
-	.cra_exit		= crypto842_exit,
-	.cra_u			= { .compress = {
-	.coa_compress		= crypto842_compress,
-	.coa_decompress		= crypto842_decompress } }
-};
-
 static struct scomp_alg scomp = {
 	.alloc_ctx		= crypto842_alloc_ctx,
 	.free_ctx		= crypto842_free_ctx,
@@ -121,25 +73,12 @@ static struct scomp_alg scomp = {
 
 static int __init crypto842_mod_init(void)
 {
-	int ret;
-
-	ret = crypto_register_alg(&alg);
-	if (ret)
-		return ret;
-
-	ret = crypto_register_scomp(&scomp);
-	if (ret) {
-		crypto_unregister_alg(&alg);
-		return ret;
-	}
-
-	return ret;
+	return crypto_register_scomp(&scomp);
 }
 subsys_initcall(crypto842_mod_init);
 
 static void __exit crypto842_mod_exit(void)
 {
-	crypto_unregister_alg(&alg);
 	crypto_unregister_scomp(&scomp);
 }
 module_exit(crypto842_mod_exit);
Signature · 1 line
2.39.2
#27 lore raw

RFC PATCH 11/21 crypto: deflate - drop obsolete 'comp' implementation

In reply to Ard Biesheuvel

No users of the obsolete 'comp' crypto compression API remain, so let's drop the software deflate version of it.

---
 crypto/deflate.c | 58 +-------------------
 1 file changed, 1 insertion(+), 57 deletions(-)
crypto/deflate.c +1 -57
index f4f127078fe2a5aa..0955040ca9e64146 100644
@@ -130,13 +130,6 @@ static void *deflate_alloc_ctx(struct crypto_scomp *tfm)
 	return ctx;
 }
 
-static int deflate_init(struct crypto_tfm *tfm)
-{
-	struct deflate_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	return __deflate_init(ctx);
-}
-
 static void __deflate_exit(void *ctx)
 {
 	deflate_comp_exit(ctx);
@@ -149,13 +142,6 @@ static void deflate_free_ctx(struct crypto_scomp *tfm, void *ctx)
 	kfree_sensitive(ctx);
 }
 
-static void deflate_exit(struct crypto_tfm *tfm)
-{
-	struct deflate_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	__deflate_exit(ctx);
-}
-
 static int __deflate_compress(const u8 *src, unsigned int slen,
 			      u8 *dst, unsigned int *dlen, void *ctx)
 {
@@ -185,14 +171,6 @@ static int __deflate_compress(const u8 *src, unsigned int slen,
 	return ret;
 }
 
-static int deflate_compress(struct crypto_tfm *tfm, const u8 *src,
-			    unsigned int slen, u8 *dst, unsigned int *dlen)
-{
-	struct deflate_ctx *dctx = crypto_tfm_ctx(tfm);
-
-	return __deflate_compress(src, slen, dst, dlen, dctx);
-}
-
 static int deflate_scompress(struct crypto_scomp *tfm, const u8 *src,
 			     unsigned int slen, u8 *dst, unsigned int *dlen,
 			     void *ctx)
@@ -241,14 +219,6 @@ static int __deflate_decompress(const u8 *src, unsigned int slen,
 	return ret;
 }
 
-static int deflate_decompress(struct crypto_tfm *tfm, const u8 *src,
-			      unsigned int slen, u8 *dst, unsigned int *dlen)
-{
-	struct deflate_ctx *dctx = crypto_tfm_ctx(tfm);
-
-	return __deflate_decompress(src, slen, dst, dlen, dctx);
-}
-
 static int deflate_sdecompress(struct crypto_scomp *tfm, const u8 *src,
 			       unsigned int slen, u8 *dst, unsigned int *dlen,
 			       void *ctx)
@@ -256,19 +226,6 @@ static int deflate_sdecompress(struct crypto_scomp *tfm, const u8 *src,
 	return __deflate_decompress(src, slen, dst, dlen, ctx);
 }
 
-static struct crypto_alg alg = {
-	.cra_name		= "deflate",
-	.cra_driver_name	= "deflate-generic",
-	.cra_flags		= CRYPTO_ALG_TYPE_COMPRESS,
-	.cra_ctxsize		= sizeof(struct deflate_ctx),
-	.cra_module		= THIS_MODULE,
-	.cra_init		= deflate_init,
-	.cra_exit		= deflate_exit,
-	.cra_u			= { .compress = {
-	.coa_compress 		= deflate_compress,
-	.coa_decompress  	= deflate_decompress } }
-};
-
 static struct scomp_alg scomp = {
 	.alloc_ctx		= deflate_alloc_ctx,
 	.free_ctx		= deflate_free_ctx,
@@ -283,24 +240,11 @@ static struct scomp_alg scomp = {
 
 static int __init deflate_mod_init(void)
 {
-	int ret;
-
-	ret = crypto_register_alg(&alg);
-	if (ret)
-		return ret;
-
-	ret = crypto_register_scomp(&scomp);
-	if (ret) {
-		crypto_unregister_alg(&alg);
-		return ret;
-	}
-
-	return ret;
+	return crypto_register_scomp(&scomp);
 }
 
 static void __exit deflate_mod_fini(void)
 {
-	crypto_unregister_alg(&alg);
 	crypto_unregister_scomp(&scomp);
 }
 
Signature · 1 line
2.39.2
#28 lore raw

RFC PATCH 12/21 crypto: lz4 - drop obsolete 'comp' implementation

In reply to Ard Biesheuvel

The 'comp' API is obsolete and will be removed, so remove this comp implementation.

---
 crypto/lz4.c | 61 +-------------------
 1 file changed, 1 insertion(+), 60 deletions(-)
crypto/lz4.c +1 -60
index 0606f8862e7872ad..c46b6cbd91ce10c0 100644
@@ -27,29 +27,11 @@ static void *lz4_alloc_ctx(struct crypto_scomp *tfm)
 	return ctx;
 }
 
-static int lz4_init(struct crypto_tfm *tfm)
-{
-	struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	ctx->lz4_comp_mem = lz4_alloc_ctx(NULL);
-	if (IS_ERR(ctx->lz4_comp_mem))
-		return -ENOMEM;
-
-	return 0;
-}
-
 static void lz4_free_ctx(struct crypto_scomp *tfm, void *ctx)
 {
 	vfree(ctx);
 }
 
-static void lz4_exit(struct crypto_tfm *tfm)
-{
-	struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	lz4_free_ctx(NULL, ctx->lz4_comp_mem);
-}
-
 static int __lz4_compress_crypto(const u8 *src, unsigned int slen,
 				 u8 *dst, unsigned int *dlen, void *ctx)
 {
@@ -70,14 +52,6 @@ static int lz4_scompress(struct crypto_scomp *tfm, const u8 *src,
 	return __lz4_compress_crypto(src, slen, dst, dlen, ctx);
 }
 
-static int lz4_compress_crypto(struct crypto_tfm *tfm, const u8 *src,
-			       unsigned int slen, u8 *dst, unsigned int *dlen)
-{
-	struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	return __lz4_compress_crypto(src, slen, dst, dlen, ctx->lz4_comp_mem);
-}
-
 static int __lz4_decompress_crypto(const u8 *src, unsigned int slen,
 				   u8 *dst, unsigned int *dlen, void *ctx)
 {
@@ -97,26 +71,6 @@ static int lz4_sdecompress(struct crypto_scomp *tfm, const u8 *src,
 	return __lz4_decompress_crypto(src, slen, dst, dlen, NULL);
 }
 
-static int lz4_decompress_crypto(struct crypto_tfm *tfm, const u8 *src,
-				 unsigned int slen, u8 *dst,
-				 unsigned int *dlen)
-{
-	return __lz4_decompress_crypto(src, slen, dst, dlen, NULL);
-}
-
-static struct crypto_alg alg_lz4 = {
-	.cra_name		= "lz4",
-	.cra_driver_name	= "lz4-generic",
-	.cra_flags		= CRYPTO_ALG_TYPE_COMPRESS,
-	.cra_ctxsize		= sizeof(struct lz4_ctx),
-	.cra_module		= THIS_MODULE,
-	.cra_init		= lz4_init,
-	.cra_exit		= lz4_exit,
-	.cra_u			= { .compress = {
-	.coa_compress		= lz4_compress_crypto,
-	.coa_decompress		= lz4_decompress_crypto } }
-};
-
 static struct scomp_alg scomp = {
 	.alloc_ctx		= lz4_alloc_ctx,
 	.free_ctx		= lz4_free_ctx,
@@ -131,24 +85,11 @@ static struct scomp_alg scomp = {
 
 static int __init lz4_mod_init(void)
 {
-	int ret;
-
-	ret = crypto_register_alg(&alg_lz4);
-	if (ret)
-		return ret;
-
-	ret = crypto_register_scomp(&scomp);
-	if (ret) {
-		crypto_unregister_alg(&alg_lz4);
-		return ret;
-	}
-
-	return ret;
+	return crypto_register_scomp(&scomp);
 }
 
 static void __exit lz4_mod_fini(void)
 {
-	crypto_unregister_alg(&alg_lz4);
 	crypto_unregister_scomp(&scomp);
 }
 
Signature · 1 line
2.39.2
#29 lore raw

RFC PATCH 13/21 crypto: lz4hc - drop obsolete 'comp' implementation

In reply to Ard Biesheuvel

The 'comp' API is obsolete and will be removed, so remove this comp implementation.

---
 crypto/lz4hc.c | 63 +-------------------
 1 file changed, 1 insertion(+), 62 deletions(-)
crypto/lz4hc.c +1 -62
index d7cc94aa2fcf42fa..5d6b13319f5e7683 100644
@@ -26,29 +26,11 @@ static void *lz4hc_alloc_ctx(struct crypto_scomp *tfm)
 	return ctx;
 }
 
-static int lz4hc_init(struct crypto_tfm *tfm)
-{
-	struct lz4hc_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	ctx->lz4hc_comp_mem = lz4hc_alloc_ctx(NULL);
-	if (IS_ERR(ctx->lz4hc_comp_mem))
-		return -ENOMEM;
-
-	return 0;
-}
-
 static void lz4hc_free_ctx(struct crypto_scomp *tfm, void *ctx)
 {
 	vfree(ctx);
 }
 
-static void lz4hc_exit(struct crypto_tfm *tfm)
-{
-	struct lz4hc_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	lz4hc_free_ctx(NULL, ctx->lz4hc_comp_mem);
-}
-
 static int __lz4hc_compress_crypto(const u8 *src, unsigned int slen,
 				   u8 *dst, unsigned int *dlen, void *ctx)
 {
@@ -69,16 +51,6 @@ static int lz4hc_scompress(struct crypto_scomp *tfm, const u8 *src,
 	return __lz4hc_compress_crypto(src, slen, dst, dlen, ctx);
 }
 
-static int lz4hc_compress_crypto(struct crypto_tfm *tfm, const u8 *src,
-				 unsigned int slen, u8 *dst,
-				 unsigned int *dlen)
-{
-	struct lz4hc_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	return __lz4hc_compress_crypto(src, slen, dst, dlen,
-					ctx->lz4hc_comp_mem);
-}
-
 static int __lz4hc_decompress_crypto(const u8 *src, unsigned int slen,
 				     u8 *dst, unsigned int *dlen, void *ctx)
 {
@@ -98,26 +70,6 @@ static int lz4hc_sdecompress(struct crypto_scomp *tfm, const u8 *src,
 	return __lz4hc_decompress_crypto(src, slen, dst, dlen, NULL);
 }
 
-static int lz4hc_decompress_crypto(struct crypto_tfm *tfm, const u8 *src,
-				   unsigned int slen, u8 *dst,
-				   unsigned int *dlen)
-{
-	return __lz4hc_decompress_crypto(src, slen, dst, dlen, NULL);
-}
-
-static struct crypto_alg alg_lz4hc = {
-	.cra_name		= "lz4hc",
-	.cra_driver_name	= "lz4hc-generic",
-	.cra_flags		= CRYPTO_ALG_TYPE_COMPRESS,
-	.cra_ctxsize		= sizeof(struct lz4hc_ctx),
-	.cra_module		= THIS_MODULE,
-	.cra_init		= lz4hc_init,
-	.cra_exit		= lz4hc_exit,
-	.cra_u			= { .compress = {
-	.coa_compress		= lz4hc_compress_crypto,
-	.coa_decompress		= lz4hc_decompress_crypto } }
-};
-
 static struct scomp_alg scomp = {
 	.alloc_ctx		= lz4hc_alloc_ctx,
 	.free_ctx		= lz4hc_free_ctx,
@@ -132,24 +84,11 @@ static struct scomp_alg scomp = {
 
 static int __init lz4hc_mod_init(void)
 {
-	int ret;
-
-	ret = crypto_register_alg(&alg_lz4hc);
-	if (ret)
-		return ret;
-
-	ret = crypto_register_scomp(&scomp);
-	if (ret) {
-		crypto_unregister_alg(&alg_lz4hc);
-		return ret;
-	}
-
-	return ret;
+	return crypto_register_scomp(&scomp);
 }
 
 static void __exit lz4hc_mod_fini(void)
 {
-	crypto_unregister_alg(&alg_lz4hc);
 	crypto_unregister_scomp(&scomp);
 }
 
Signature · 1 line
2.39.2
#30 lore raw

RFC PATCH 14/21 crypto: lzo-rle - drop obsolete 'comp' implementation

In reply to Ard Biesheuvel

The 'comp' API is obsolete and will be removed, so remove this comp implementation.

---
 crypto/lzo-rle.c | 60 +-------------------
 1 file changed, 1 insertion(+), 59 deletions(-)
crypto/lzo-rle.c +1 -59
index 0631d975bfac1129..658d6aa46fe21e19 100644
@@ -26,29 +26,11 @@ static void *lzorle_alloc_ctx(struct crypto_scomp *tfm)
 	return ctx;
 }
 
-static int lzorle_init(struct crypto_tfm *tfm)
-{
-	struct lzorle_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	ctx->lzorle_comp_mem = lzorle_alloc_ctx(NULL);
-	if (IS_ERR(ctx->lzorle_comp_mem))
-		return -ENOMEM;
-
-	return 0;
-}
-
 static void lzorle_free_ctx(struct crypto_scomp *tfm, void *ctx)
 {
 	kvfree(ctx);
 }
 
-static void lzorle_exit(struct crypto_tfm *tfm)
-{
-	struct lzorle_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	lzorle_free_ctx(NULL, ctx->lzorle_comp_mem);
-}
-
 static int __lzorle_compress(const u8 *src, unsigned int slen,
 			  u8 *dst, unsigned int *dlen, void *ctx)
 {
@@ -64,14 +46,6 @@ static int __lzorle_compress(const u8 *src, unsigned int slen,
 	return 0;
 }
 
-static int lzorle_compress(struct crypto_tfm *tfm, const u8 *src,
-			unsigned int slen, u8 *dst, unsigned int *dlen)
-{
-	struct lzorle_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	return __lzorle_compress(src, slen, dst, dlen, ctx->lzorle_comp_mem);
-}
-
 static int lzorle_scompress(struct crypto_scomp *tfm, const u8 *src,
 			 unsigned int slen, u8 *dst, unsigned int *dlen,
 			 void *ctx)
@@ -94,12 +68,6 @@ static int __lzorle_decompress(const u8 *src, unsigned int slen,
 	return 0;
 }
 
-static int lzorle_decompress(struct crypto_tfm *tfm, const u8 *src,
-			  unsigned int slen, u8 *dst, unsigned int *dlen)
-{
-	return __lzorle_decompress(src, slen, dst, dlen);
-}
-
 static int lzorle_sdecompress(struct crypto_scomp *tfm, const u8 *src,
 			   unsigned int slen, u8 *dst, unsigned int *dlen,
 			   void *ctx)
@@ -107,19 +75,6 @@ static int lzorle_sdecompress(struct crypto_scomp *tfm, const u8 *src,
 	return __lzorle_decompress(src, slen, dst, dlen);
 }
 
-static struct crypto_alg alg = {
-	.cra_name		= "lzo-rle",
-	.cra_driver_name	= "lzo-rle-generic",
-	.cra_flags		= CRYPTO_ALG_TYPE_COMPRESS,
-	.cra_ctxsize		= sizeof(struct lzorle_ctx),
-	.cra_module		= THIS_MODULE,
-	.cra_init		= lzorle_init,
-	.cra_exit		= lzorle_exit,
-	.cra_u			= { .compress = {
-	.coa_compress		= lzorle_compress,
-	.coa_decompress		= lzorle_decompress } }
-};
-
 static struct scomp_alg scomp = {
 	.alloc_ctx		= lzorle_alloc_ctx,
 	.free_ctx		= lzorle_free_ctx,
@@ -134,24 +89,11 @@ static struct scomp_alg scomp = {
 
 static int __init lzorle_mod_init(void)
 {
-	int ret;
-
-	ret = crypto_register_alg(&alg);
-	if (ret)
-		return ret;
-
-	ret = crypto_register_scomp(&scomp);
-	if (ret) {
-		crypto_unregister_alg(&alg);
-		return ret;
-	}
-
-	return ret;
+	return crypto_register_scomp(&scomp);
 }
 
 static void __exit lzorle_mod_fini(void)
 {
-	crypto_unregister_alg(&alg);
 	crypto_unregister_scomp(&scomp);
 }
 
Signature · 1 line
2.39.2
#31 lore raw

RFC PATCH 15/21 crypto: lzo - drop obsolete 'comp' implementation

In reply to Ard Biesheuvel

The 'comp' API is obsolete and will be removed, so remove this comp implementation.

---
 crypto/lzo.c | 60 +-------------------
 1 file changed, 1 insertion(+), 59 deletions(-)
crypto/lzo.c +1 -59
index ebda132dd22bf543..52558f9d41f3dcea 100644
@@ -26,29 +26,11 @@ static void *lzo_alloc_ctx(struct crypto_scomp *tfm)
 	return ctx;
 }
 
-static int lzo_init(struct crypto_tfm *tfm)
-{
-	struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	ctx->lzo_comp_mem = lzo_alloc_ctx(NULL);
-	if (IS_ERR(ctx->lzo_comp_mem))
-		return -ENOMEM;
-
-	return 0;
-}
-
 static void lzo_free_ctx(struct crypto_scomp *tfm, void *ctx)
 {
 	kvfree(ctx);
 }
 
-static void lzo_exit(struct crypto_tfm *tfm)
-{
-	struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	lzo_free_ctx(NULL, ctx->lzo_comp_mem);
-}
-
 static int __lzo_compress(const u8 *src, unsigned int slen,
 			  u8 *dst, unsigned int *dlen, void *ctx)
 {
@@ -64,14 +46,6 @@ static int __lzo_compress(const u8 *src, unsigned int slen,
 	return 0;
 }
 
-static int lzo_compress(struct crypto_tfm *tfm, const u8 *src,
-			unsigned int slen, u8 *dst, unsigned int *dlen)
-{
-	struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	return __lzo_compress(src, slen, dst, dlen, ctx->lzo_comp_mem);
-}
-
 static int lzo_scompress(struct crypto_scomp *tfm, const u8 *src,
 			 unsigned int slen, u8 *dst, unsigned int *dlen,
 			 void *ctx)
@@ -94,12 +68,6 @@ static int __lzo_decompress(const u8 *src, unsigned int slen,
 	return 0;
 }
 
-static int lzo_decompress(struct crypto_tfm *tfm, const u8 *src,
-			  unsigned int slen, u8 *dst, unsigned int *dlen)
-{
-	return __lzo_decompress(src, slen, dst, dlen);
-}
-
 static int lzo_sdecompress(struct crypto_scomp *tfm, const u8 *src,
 			   unsigned int slen, u8 *dst, unsigned int *dlen,
 			   void *ctx)
@@ -107,19 +75,6 @@ static int lzo_sdecompress(struct crypto_scomp *tfm, const u8 *src,
 	return __lzo_decompress(src, slen, dst, dlen);
 }
 
-static struct crypto_alg alg = {
-	.cra_name		= "lzo",
-	.cra_driver_name	= "lzo-generic",
-	.cra_flags		= CRYPTO_ALG_TYPE_COMPRESS,
-	.cra_ctxsize		= sizeof(struct lzo_ctx),
-	.cra_module		= THIS_MODULE,
-	.cra_init		= lzo_init,
-	.cra_exit		= lzo_exit,
-	.cra_u			= { .compress = {
-	.coa_compress		= lzo_compress,
-	.coa_decompress		= lzo_decompress } }
-};
-
 static struct scomp_alg scomp = {
 	.alloc_ctx		= lzo_alloc_ctx,
 	.free_ctx		= lzo_free_ctx,
@@ -134,24 +89,11 @@ static struct scomp_alg scomp = {
 
 static int __init lzo_mod_init(void)
 {
-	int ret;
-
-	ret = crypto_register_alg(&alg);
-	if (ret)
-		return ret;
-
-	ret = crypto_register_scomp(&scomp);
-	if (ret) {
-		crypto_unregister_alg(&alg);
-		return ret;
-	}
-
-	return ret;
+	return crypto_register_scomp(&scomp);
 }
 
 static void __exit lzo_mod_fini(void)
 {
-	crypto_unregister_alg(&alg);
 	crypto_unregister_scomp(&scomp);
 }
 
Signature · 1 line
2.39.2
#32 lore raw

RFC PATCH 16/21 crypto: zstd - drop obsolete 'comp' implementation

In reply to Ard Biesheuvel

The 'comp' API is obsolete and will be removed, so remove this comp implementation.

---
 crypto/zstd.c | 56 +-------------------
 1 file changed, 1 insertion(+), 55 deletions(-)
crypto/zstd.c +1 -55
index 154a969c83a82277..c6e6f135c5812c9c 100644
@@ -121,13 +121,6 @@ static void *zstd_alloc_ctx(struct crypto_scomp *tfm)
 	return ctx;
 }
 
-static int zstd_init(struct crypto_tfm *tfm)
-{
-	struct zstd_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	return __zstd_init(ctx);
-}
-
 static void __zstd_exit(void *ctx)
 {
 	zstd_comp_exit(ctx);
@@ -140,13 +133,6 @@ static void zstd_free_ctx(struct crypto_scomp *tfm, void *ctx)
 	kfree_sensitive(ctx);
 }
 
-static void zstd_exit(struct crypto_tfm *tfm)
-{
-	struct zstd_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	__zstd_exit(ctx);
-}
-
 static int __zstd_compress(const u8 *src, unsigned int slen,
 			   u8 *dst, unsigned int *dlen, void *ctx)
 {
@@ -161,14 +147,6 @@ static int __zstd_compress(const u8 *src, unsigned int slen,
 	return 0;
 }
 
-static int zstd_compress(struct crypto_tfm *tfm, const u8 *src,
-			 unsigned int slen, u8 *dst, unsigned int *dlen)
-{
-	struct zstd_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	return __zstd_compress(src, slen, dst, dlen, ctx);
-}
-
 static int zstd_scompress(struct crypto_scomp *tfm, const u8 *src,
 			  unsigned int slen, u8 *dst, unsigned int *dlen,
 			  void *ctx)
@@ -189,14 +167,6 @@ static int __zstd_decompress(const u8 *src, unsigned int slen,
 	return 0;
 }
 
-static int zstd_decompress(struct crypto_tfm *tfm, const u8 *src,
-			   unsigned int slen, u8 *dst, unsigned int *dlen)
-{
-	struct zstd_ctx *ctx = crypto_tfm_ctx(tfm);
-
-	return __zstd_decompress(src, slen, dst, dlen, ctx);
-}
-
 static int zstd_sdecompress(struct crypto_scomp *tfm, const u8 *src,
 			    unsigned int slen, u8 *dst, unsigned int *dlen,
 			    void *ctx)
@@ -204,19 +174,6 @@ static int zstd_sdecompress(struct crypto_scomp *tfm, const u8 *src,
 	return __zstd_decompress(src, slen, dst, dlen, ctx);
 }
 
-static struct crypto_alg alg = {
-	.cra_name		= "zstd",
-	.cra_driver_name	= "zstd-generic",
-	.cra_flags		= CRYPTO_ALG_TYPE_COMPRESS,
-	.cra_ctxsize		= sizeof(struct zstd_ctx),
-	.cra_module		= THIS_MODULE,
-	.cra_init		= zstd_init,
-	.cra_exit		= zstd_exit,
-	.cra_u			= { .compress = {
-	.coa_compress		= zstd_compress,
-	.coa_decompress		= zstd_decompress } }
-};
-
 static struct scomp_alg scomp = {
 	.alloc_ctx		= zstd_alloc_ctx,
 	.free_ctx		= zstd_free_ctx,
@@ -231,22 +188,11 @@ static struct scomp_alg scomp = {
 
 static int __init zstd_mod_init(void)
 {
-	int ret;
-
-	ret = crypto_register_alg(&alg);
-	if (ret)
-		return ret;
-
-	ret = crypto_register_scomp(&scomp);
-	if (ret)
-		crypto_unregister_alg(&alg);
-
-	return ret;
+	return crypto_register_scomp(&scomp);
 }
 
 static void __exit zstd_mod_fini(void)
 {
-	crypto_unregister_alg(&alg);
 	crypto_unregister_scomp(&scomp);
 }
 
Signature · 1 line
2.39.2
#33 lore raw

RFC PATCH 17/21 crypto: cavium/zip - drop obsolete 'comp' implementation

In reply to Ard Biesheuvel

The 'comp' API is obsolete and will be removed, so remove this comp implementation.

---
 drivers/crypto/cavium/zip/zip_crypto.c | 40 ----------------
 drivers/crypto/cavium/zip/zip_crypto.h | 10 ----
 drivers/crypto/cavium/zip/zip_main.c   | 50 +-------------------
 3 files changed, 1 insertion(+), 99 deletions(-)
drivers/crypto/cavium/zip/zip_crypto.c +0 -40
index 1046a746d36f551c..5edad3b1d1dc8398 100644
@@ -195,46 +195,6 @@ static int zip_decompress(const u8 *src, unsigned int slen,
 	return ret;
 }
 
-/* Legacy Compress framework start */
-int zip_alloc_comp_ctx_deflate(struct crypto_tfm *tfm)
-{
-	struct zip_kernel_ctx *zip_ctx = crypto_tfm_ctx(tfm);
-
-	return zip_ctx_init(zip_ctx, 0);
-}
-
-int zip_alloc_comp_ctx_lzs(struct crypto_tfm *tfm)
-{
-	struct zip_kernel_ctx *zip_ctx = crypto_tfm_ctx(tfm);
-
-	return zip_ctx_init(zip_ctx, 1);
-}
-
-void zip_free_comp_ctx(struct crypto_tfm *tfm)
-{
-	struct zip_kernel_ctx *zip_ctx = crypto_tfm_ctx(tfm);
-
-	zip_ctx_exit(zip_ctx);
-}
-
-int  zip_comp_compress(struct crypto_tfm *tfm,
-		       const u8 *src, unsigned int slen,
-		       u8 *dst, unsigned int *dlen)
-{
-	struct zip_kernel_ctx *zip_ctx = crypto_tfm_ctx(tfm);
-
-	return zip_compress(src, slen, dst, dlen, zip_ctx);
-}
-
-int  zip_comp_decompress(struct crypto_tfm *tfm,
-			 const u8 *src, unsigned int slen,
-			 u8 *dst, unsigned int *dlen)
-{
-	struct zip_kernel_ctx *zip_ctx = crypto_tfm_ctx(tfm);
-
-	return zip_decompress(src, slen, dst, dlen, zip_ctx);
-} /* Legacy compress framework end */
-
 /* SCOMP framework start */
 void *zip_alloc_scomp_ctx_deflate(struct crypto_scomp *tfm)
 {
drivers/crypto/cavium/zip/zip_crypto.h +0 -10
index b59ddfcacd34447e..a1ae3825fb65c3b6 100644
@@ -57,16 +57,6 @@ struct zip_kernel_ctx {
 	struct zip_operation zip_decomp;
 };
 
-int  zip_alloc_comp_ctx_deflate(struct crypto_tfm *tfm);
-int  zip_alloc_comp_ctx_lzs(struct crypto_tfm *tfm);
-void zip_free_comp_ctx(struct crypto_tfm *tfm);
-int  zip_comp_compress(struct crypto_tfm *tfm,
-		       const u8 *src, unsigned int slen,
-		       u8 *dst, unsigned int *dlen);
-int  zip_comp_decompress(struct crypto_tfm *tfm,
-			 const u8 *src, unsigned int slen,
-			 u8 *dst, unsigned int *dlen);
-
 void *zip_alloc_scomp_ctx_deflate(struct crypto_scomp *tfm);
 void *zip_alloc_scomp_ctx_lzs(struct crypto_scomp *tfm);
 void  zip_free_scomp_ctx(struct crypto_scomp *tfm, void *zip_ctx);
drivers/crypto/cavium/zip/zip_main.c +1 -49

77 lines not on this page · raw message

Signature · 1 line
2.39.2
#34 lore raw

RFC PATCH 18/21 crypto: compress_null - drop obsolete 'comp' implementation

In reply to Ard Biesheuvel

The 'comp' API is obsolete and will be removed, so remove this comp implementation.

---
 crypto/crypto_null.c | 31 ++++----------------
 crypto/testmgr.c     |  3 --
 2 files changed, 5 insertions(+), 29 deletions(-)
crypto/crypto_null.c +5 -26
index 5b84b0f7cc178fcd..75e73b1d6df01cc6 100644
@@ -24,16 +24,6 @@ static DEFINE_MUTEX(crypto_default_null_skcipher_lock);
 static struct crypto_sync_skcipher *crypto_default_null_skcipher;
 static int crypto_default_null_skcipher_refcnt;
 
-static int null_compress(struct crypto_tfm *tfm, const u8 *src,
-			 unsigned int slen, u8 *dst, unsigned int *dlen)
-{
-	if (slen > *dlen)
-		return -EINVAL;
-	memcpy(dst, src, slen);
-	*dlen = slen;
-	return 0;
-}
-
 static int null_init(struct shash_desc *desc)
 {
 	return 0;
@@ -121,7 +111,7 @@ static struct skcipher_alg skcipher_null = {
 	.decrypt		=	null_skcipher_crypt,
 };
 
-static struct crypto_alg null_algs[] = { {
+static struct crypto_alg cipher_null = {
 	.cra_name		=	"cipher_null",
 	.cra_driver_name	=	"cipher_null-generic",
 	.cra_flags		=	CRYPTO_ALG_TYPE_CIPHER,
@@ -134,19 +124,8 @@ static struct crypto_alg null_algs[] = { {
 	.cia_setkey		= 	null_setkey,
 	.cia_encrypt		=	null_crypt,
 	.cia_decrypt		=	null_crypt } }
-}, {
-	.cra_name		=	"compress_null",
-	.cra_driver_name	=	"compress_null-generic",
-	.cra_flags		=	CRYPTO_ALG_TYPE_COMPRESS,
-	.cra_blocksize		=	NULL_BLOCK_SIZE,
-	.cra_ctxsize		=	0,
-	.cra_module		=	THIS_MODULE,
-	.cra_u			=	{ .compress = {
-	.coa_compress		=	null_compress,
-	.coa_decompress		=	null_compress } }
-} };
+};
 
-MODULE_ALIAS_CRYPTO("compress_null");
 MODULE_ALIAS_CRYPTO("digest_null");
 MODULE_ALIAS_CRYPTO("cipher_null");
 
@@ -189,7 +168,7 @@ static int __init crypto_null_mod_init(void)
 {
 	int ret = 0;
 
-	ret = crypto_register_algs(null_algs, ARRAY_SIZE(null_algs));
+	ret = crypto_register_alg(&cipher_null);
 	if (ret < 0)
 		goto out;
 
@@ -206,14 +185,14 @@ static int __init crypto_null_mod_init(void)
 out_unregister_shash:
 	crypto_unregister_shash(&digest_null);
 out_unregister_algs:
-	crypto_unregister_algs(null_algs, ARRAY_SIZE(null_algs));
+	crypto_unregister_alg(&cipher_null);
 out:
 	return ret;
 }
 
 static void __exit crypto_null_mod_fini(void)
 {
-	crypto_unregister_algs(null_algs, ARRAY_SIZE(null_algs));
+	crypto_unregister_alg(&cipher_null);
 	crypto_unregister_shash(&digest_null);
 	crypto_unregister_skcipher(&skcipher_null);
 }
crypto/testmgr.c +0 -3
index 4971351f55dbabb9..e4b6d67233763193 100644
@@ -4633,9 +4633,6 @@ static const struct alg_test_desc alg_test_descs[] = {
 		.suite = {
 			.hash = __VECS(sm4_cmac128_tv_template)
 		}
-	}, {
-		.alg = "compress_null",
-		.test = alg_test_null,
 	}, {
 		.alg = "crc32",
 		.test = alg_test_hash,
Signature · 1 line
2.39.2
#35 lore raw

RFC PATCH 19/21 crypto: remove obsolete 'comp' compression API

In reply to Ard Biesheuvel

The 'comp' compression API has been superseded by the acomp API, which is a bit more cumbersome to use, but ultimately more flexible when it comes to hardware implementations.

Now that all the users and implementations have been removed, let's remove the core plumbing of the 'comp' API as well.

---
 Documentation/crypto/architecture.rst |   2 -
 crypto/Makefile                       |   2 +-
 crypto/api.c                          |   4 -
 crypto/compress.c                     |  32 -----
 crypto/crypto_user_base.c             |  16 ---
 crypto/crypto_user_stat.c             |   4 -
 crypto/proc.c                         |   3 -
 crypto/testmgr.c                      | 144 ++------------------
 include/linux/crypto.h                |  49 +------
 9 files changed, 12 insertions(+), 244 deletions(-)
Documentation/crypto/architecture.rst +0 -2
index 646c3380a7edc4c6..ec7436aade15c2e6 100644
@@ -196,8 +196,6 @@ the aforementioned cipher types:
 
 -  CRYPTO_ALG_TYPE_CIPHER Single block cipher
 
--  CRYPTO_ALG_TYPE_COMPRESS Compression
-
 -  CRYPTO_ALG_TYPE_AEAD Authenticated Encryption with Associated Data
    (MAC)
 
crypto/Makefile +1 -1
index 953a7e105e58c837..5775440c62e09eac 100644
@@ -4,7 +4,7 @@
 #
 
 obj-$(CONFIG_CRYPTO) += crypto.o
-crypto-y := api.o cipher.o compress.o
+crypto-y := api.o cipher.o
 
 obj-$(CONFIG_CRYPTO_ENGINE) += crypto_engine.o
 obj-$(CONFIG_CRYPTO_FIPS) += fips.o
crypto/api.c +0 -4
index b9cc0c906efe0706..23d691a70bc3fb00 100644
@@ -369,10 +369,6 @@ static unsigned int crypto_ctxsize(struct crypto_alg *alg, u32 type, u32 mask)
 	case CRYPTO_ALG_TYPE_CIPHER:
 		len += crypto_cipher_ctxsize(alg);
 		break;
-
-	case CRYPTO_ALG_TYPE_COMPRESS:
-		len += crypto_compress_ctxsize(alg);
-		break;
 	}
 
 	return len;
crypto/compress.c deleted +0 -32
deleted file mode 100644
index 9048fe390c463069..0000000000000000
@@ -1,32 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Cryptographic API.
- *
- * Compression operations.
- *
- * Copyright (c) 2002 James Morris <[email protected]>
- */
-#include <linux/crypto.h>
-#include "internal.h"
-
-int crypto_comp_compress(struct crypto_comp *comp,
-			 const u8 *src, unsigned int slen,
-			 u8 *dst, unsigned int *dlen)
-{
-	struct crypto_tfm *tfm = crypto_comp_tfm(comp);
-
-	return tfm->__crt_alg->cra_compress.coa_compress(tfm, src, slen, dst,
-	                                                 dlen);
-}
-EXPORT_SYMBOL_GPL(crypto_comp_compress);
-
-int crypto_comp_decompress(struct crypto_comp *comp,
-			   const u8 *src, unsigned int slen,
-			   u8 *dst, unsigned int *dlen)
-{
-	struct crypto_tfm *tfm = crypto_comp_tfm(comp);
-
-	return tfm->__crt_alg->cra_compress.coa_decompress(tfm, src, slen, dst,
-	                                                   dlen);
-}
-EXPORT_SYMBOL_GPL(crypto_comp_decompress);
crypto/crypto_user_base.c +0 -16
index 3fa20f12989f7ef2..c27484b0042e6bd8 100644
@@ -85,17 +85,6 @@ static int crypto_report_cipher(struct sk_buff *skb, struct crypto_alg *alg)
 		       sizeof(rcipher), &rcipher);
 }
 
-static int crypto_report_comp(struct sk_buff *skb, struct crypto_alg *alg)
-{
-	struct crypto_report_comp rcomp;
-
-	memset(&rcomp, 0, sizeof(rcomp));
-
-	strscpy(rcomp.type, "compression", sizeof(rcomp.type));
-
-	return nla_put(skb, CRYPTOCFGA_REPORT_COMPRESS, sizeof(rcomp), &rcomp);
-}
-
 static int crypto_report_one(struct crypto_alg *alg,
 			     struct crypto_user_alg *ualg, struct sk_buff *skb)
 {
@@ -136,11 +125,6 @@ static int crypto_report_one(struct crypto_alg *alg,
 		if (crypto_report_cipher(skb, alg))
 			goto nla_put_failure;
 
-		break;
-	case CRYPTO_ALG_TYPE_COMPRESS:
-		if (crypto_report_comp(skb, alg))
-			goto nla_put_failure;
-
 		break;
 	}
 
crypto/crypto_user_stat.c +0 -4
index d4f3d39b51376973..d3133eda2f528d17 100644
@@ -86,10 +86,6 @@ static int crypto_reportstat_one(struct crypto_alg *alg,
 		if (crypto_report_cipher(skb, alg))
 			goto nla_put_failure;
 		break;
-	case CRYPTO_ALG_TYPE_COMPRESS:
-		if (crypto_report_comp(skb, alg))
-			goto nla_put_failure;
-		break;
 	default:
 		pr_err("ERROR: Unhandled alg %d in %s\n",
 		       alg->cra_flags & (CRYPTO_ALG_TYPE_MASK | CRYPTO_ALG_LARVAL),
crypto/proc.c +0 -3
index 56c7c78df29713e3..88cc9830a9a907a8 100644
@@ -75,9 +75,6 @@ static int c_show(struct seq_file *m, void *p)
 		seq_printf(m, "max keysize  : %u\n",
 					alg->cra_cipher.cia_max_keysize);
 		break;
-	case CRYPTO_ALG_TYPE_COMPRESS:
-		seq_printf(m, "type         : compression\n");
-		break;
 	default:
 		seq_printf(m, "type         : unknown\n");
 		break;
crypto/testmgr.c +10 -134

160 lines not on this page · raw message

include/linux/crypto.h +1 -48

60 lines not on this page · raw message

Signature · 1 line
2.39.2
#36 lore raw

Re: [RFC PATCH 19/21] crypto: remove obsolete 'comp' compression API

On Tue, Jul 18, 2023 at 02:58:45PM +0200, Ard Biesheuvel wrote:

...

crypto/crypto_user_stat.c static int crypto_reportstat_one(struct crypto_alg *alg, · line 92

Quoted 12 lines
crypto/crypto_user_stat.c +0 -4
index d4f3d39b51376973..d3133eda2f528d17 100644
@@ -86,10 +86,6 @@ static int crypto_reportstat_one(struct crypto_alg *alg,
 		if (crypto_report_cipher(skb, alg))
 			goto nla_put_failure;
 		break;
-	case CRYPTO_ALG_TYPE_COMPRESS:
-		if (crypto_report_comp(skb, alg))
-			goto nla_put_failure;
-		break;

Hi Ard,

It seems that there is an implementation of crypto_report_comp() in this file which is now unused and can be removed.

Quoted 3 lines
crypto/crypto_user_stat.c +0 -0
 	default:
 		pr_err("ERROR: Unhandled alg %d in %s\n",
 		       alg->cra_flags & (CRYPTO_ALG_TYPE_MASK | CRYPTO_ALG_LARVAL),

...

#37 lore raw

RFC PATCH 20/21 crypto: deflate - implement acomp API directly

In reply to Ard Biesheuvel

Drop the scomp implementation of deflate, which can only operate on contiguous in- and output buffer, and replace it with an implementation of acomp directly. This implementation walks the scatterlists, removing the need for the caller to use scratch buffers to present the input and output in a contiguous manner.

This is intended for use by the IPcomp code, which currently needs to 'linearize' SKBs in order for the compression to be able to consume the input in a single chunk.

---
 crypto/deflate.c             | 315 +++++++-------------
 include/crypto/scatterwalk.h |   2 +-
 2 files changed, 113 insertions(+), 204 deletions(-)
crypto/deflate.c +112 -203
index 0955040ca9e64146..112683473df2b588 100644
@@ -6,246 +6,154 @@
  * by IPCOMP (RFC 3173 & RFC 2394).
  *
  * Copyright (c) 2003 James Morris <[email protected]>
- *
- * FIXME: deflate transforms will require up to a total of about 436k of kernel
- * memory on i386 (390k for compression, the rest for decompression), as the
- * current zlib kernel code uses a worst case pre-allocation system by default.
- * This needs to be fixed so that the amount of memory required is properly
- * related to the  winbits and memlevel parameters.
- *
- * The default winbits of 11 should suit most packets, and it may be something
- * to configure on a per-tfm basis in the future.
- *
- * Currently, compression history is not maintained between tfm calls, as
- * it is not needed for IPCOMP and keeps the code simpler.  It can be
- * implemented if someone wants it.
+ * Copyright (c) 2023 Google, LLC. <[email protected]>
  */
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/crypto.h>
 #include <linux/zlib.h>
-#include <linux/vmalloc.h>
-#include <linux/interrupt.h>
-#include <linux/mm.h>
 #include <linux/net.h>
-#include <crypto/internal/scompress.h>
+#include <linux/scatterlist.h>
+#include <crypto/scatterwalk.h>
+#include <crypto/internal/acompress.h>
 
 #define DEFLATE_DEF_LEVEL		Z_DEFAULT_COMPRESSION
 #define DEFLATE_DEF_WINBITS		11
 #define DEFLATE_DEF_MEMLEVEL		MAX_MEM_LEVEL
 
-struct deflate_ctx {
-	struct z_stream_s comp_stream;
-	struct z_stream_s decomp_stream;
+struct deflate_req_ctx {
+	struct z_stream_s stream;
+	u8 workspace[];
 };
 
-static int deflate_comp_init(struct deflate_ctx *ctx)
+static int deflate_process(struct acomp_req *req, struct z_stream_s *stream,
+			   int (*process)(struct z_stream_s *, int))
 {
-	int ret = 0;
-	struct z_stream_s *stream = &ctx->comp_stream;
+	unsigned int slen = req->slen;
+	unsigned int dlen = req->dlen;
+	struct scatter_walk src, dst;
+	unsigned int scur, dcur;
+	int ret;
 
-	stream->workspace = vzalloc(zlib_deflate_workspacesize(
-				-DEFLATE_DEF_WINBITS, DEFLATE_DEF_MEMLEVEL));
-	if (!stream->workspace) {
-		ret = -ENOMEM;
-		goto out;
-	}
+	stream->avail_in = stream->avail_out = 0;
+
+	scatterwalk_start(&src, req->src);
+	scatterwalk_start(&dst, req->dst);
+
+	scur = dcur = 0;
+
+	do {
+		if (stream->avail_in == 0) {
+			if (scur) {
+				slen -= scur;
+
+				scatterwalk_unmap(stream->next_in - scur);
+				scatterwalk_advance(&src, scur);
+				scatterwalk_done(&src, 0, slen);
+			}
+
+			scur = scatterwalk_clamp(&src, slen);
+			if (scur) {
+				stream->next_in = scatterwalk_map(&src);
+				stream->avail_in = scur;
+			}
+		}
+
+		if (stream->avail_out == 0) {
+			if (dcur) {
+				dlen -= dcur;
+
+				scatterwalk_unmap(stream->next_out - dcur);
+				scatterwalk_advance(&dst, dcur);
+				scatterwalk_done(&dst, 1, dlen);
+			}
+
+			dcur = scatterwalk_clamp(&dst, dlen);
+			if (!dcur)
+				break;
+
+			stream->next_out = scatterwalk_map(&dst);
+			stream->avail_out = dcur;
+		}
+
+		ret = process(stream, (slen == scur) ? Z_FINISH : Z_SYNC_FLUSH);
+	} while (ret == Z_OK);
+
+	if (scur)
+		scatterwalk_unmap(stream->next_in - scur);
+	if (dcur)
+		scatterwalk_unmap(stream->next_out - dcur);
+
+	if (ret != Z_STREAM_END)
+		return -EINVAL;
+
+	req->dlen = stream->total_out;
+	return 0;
+}
+
+static int deflate_compress(struct acomp_req *req)
+{
+	struct deflate_req_ctx *ctx = acomp_request_ctx(req);
+	struct z_stream_s *stream = &ctx->stream;
+	int ret;
+
+        if (!req->src || !req->slen || !req->dst || !req->dlen)
+                return -EINVAL;
+
+	stream->workspace = ctx->workspace;
 	ret = zlib_deflateInit2(stream, DEFLATE_DEF_LEVEL, Z_DEFLATED,
 	                        -DEFLATE_DEF_WINBITS, DEFLATE_DEF_MEMLEVEL,
 	                        Z_DEFAULT_STRATEGY);
-	if (ret != Z_OK) {
-		ret = -EINVAL;
-		goto out_free;
-	}
-out:
+	if (ret != Z_OK)
+		return -EINVAL;
+
+	ret = deflate_process(req, stream, zlib_deflate);
+	zlib_deflateEnd(stream);
 	return ret;
-out_free:
-	vfree(stream->workspace);
-	goto out;
 }
 
-static int deflate_decomp_init(struct deflate_ctx *ctx)
+static int deflate_decompress(struct acomp_req *req)
 {
-	int ret = 0;
-	struct z_stream_s *stream = &ctx->decomp_stream;
+	struct deflate_req_ctx *ctx = acomp_request_ctx(req);
+	struct z_stream_s *stream = &ctx->stream;
+	int ret;
 
-	stream->workspace = vzalloc(zlib_inflate_workspacesize());
-	if (!stream->workspace) {
-		ret = -ENOMEM;
-		goto out;
-	}
+        if (!req->src || !req->slen || !req->dst || !req->dlen)
+                return -EINVAL;
+
+	stream->workspace = ctx->workspace;
 	ret = zlib_inflateInit2(stream, -DEFLATE_DEF_WINBITS);
-	if (ret != Z_OK) {
-		ret = -EINVAL;
-		goto out_free;
-	}
-out:
-	return ret;
-out_free:
-	vfree(stream->workspace);
-	goto out;
-}
+	if (ret != Z_OK)
+		return -EINVAL;
 
-static void deflate_comp_exit(struct deflate_ctx *ctx)
-{
-	zlib_deflateEnd(&ctx->comp_stream);
-	vfree(ctx->comp_stream.workspace);
-}
-
-static void deflate_decomp_exit(struct deflate_ctx *ctx)
-{
-	zlib_inflateEnd(&ctx->decomp_stream);
-	vfree(ctx->decomp_stream.workspace);
-}
-
-static int __deflate_init(void *ctx)
-{
-	int ret;
-
-	ret = deflate_comp_init(ctx);
-	if (ret)
-		goto out;
-	ret = deflate_decomp_init(ctx);
-	if (ret)
-		deflate_comp_exit(ctx);
-out:
+	ret = deflate_process(req, stream, zlib_inflate);
+	req->dlen = stream->total_out;
+	zlib_inflateEnd(stream);
 	return ret;
 }
 
-static void *deflate_alloc_ctx(struct crypto_scomp *tfm)
-{
-	struct deflate_ctx *ctx;
-	int ret;
+static struct acomp_alg alg = {
+	.compress		= deflate_compress,
+	.decompress		= deflate_decompress,
 
-	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
-	if (!ctx)
-		return ERR_PTR(-ENOMEM);
-
-	ret = __deflate_init(ctx);
-	if (ret) {
-		kfree(ctx);
-		return ERR_PTR(ret);
-	}
-
-	return ctx;
-}
-
-static void __deflate_exit(void *ctx)
-{
-	deflate_comp_exit(ctx);
-	deflate_decomp_exit(ctx);
-}
-
-static void deflate_free_ctx(struct crypto_scomp *tfm, void *ctx)
-{
-	__deflate_exit(ctx);
-	kfree_sensitive(ctx);
-}
-
-static int __deflate_compress(const u8 *src, unsigned int slen,
-			      u8 *dst, unsigned int *dlen, void *ctx)
-{
-	int ret = 0;
-	struct deflate_ctx *dctx = ctx;
-	struct z_stream_s *stream = &dctx->comp_stream;
-
-	ret = zlib_deflateReset(stream);
-	if (ret != Z_OK) {
-		ret = -EINVAL;
-		goto out;
-	}
-
-	stream->next_in = (u8 *)src;
-	stream->avail_in = slen;
-	stream->next_out = (u8 *)dst;
-	stream->avail_out = *dlen;
-
-	ret = zlib_deflate(stream, Z_FINISH);
-	if (ret != Z_STREAM_END) {
-		ret = -EINVAL;
-		goto out;
-	}
-	ret = 0;
-	*dlen = stream->total_out;
-out:
-	return ret;
-}
-
-static int deflate_scompress(struct crypto_scomp *tfm, const u8 *src,
-			     unsigned int slen, u8 *dst, unsigned int *dlen,
-			     void *ctx)
-{
-	return __deflate_compress(src, slen, dst, dlen, ctx);
-}
-
-static int __deflate_decompress(const u8 *src, unsigned int slen,
-				u8 *dst, unsigned int *dlen, void *ctx)
-{
-
-	int ret = 0;
-	struct deflate_ctx *dctx = ctx;
-	struct z_stream_s *stream = &dctx->decomp_stream;
-
-	ret = zlib_inflateReset(stream);
-	if (ret != Z_OK) {
-		ret = -EINVAL;
-		goto out;
-	}
-
-	stream->next_in = (u8 *)src;
-	stream->avail_in = slen;
-	stream->next_out = (u8 *)dst;
-	stream->avail_out = *dlen;
-
-	ret = zlib_inflate(stream, Z_SYNC_FLUSH);
-	/*
-	 * Work around a bug in zlib, which sometimes wants to taste an extra
-	 * byte when being used in the (undocumented) raw deflate mode.
-	 * (From USAGI).
-	 */
-	if (ret == Z_OK && !stream->avail_in && stream->avail_out) {
-		u8 zerostuff = 0;
-		stream->next_in = &zerostuff;
-		stream->avail_in = 1;
-		ret = zlib_inflate(stream, Z_FINISH);
-	}
-	if (ret != Z_STREAM_END) {
-		ret = -EINVAL;
-		goto out;
-	}
-	ret = 0;
-	*dlen = stream->total_out;
-out:
-	return ret;
-}
-
-static int deflate_sdecompress(struct crypto_scomp *tfm, const u8 *src,
-			       unsigned int slen, u8 *dst, unsigned int *dlen,
-			       void *ctx)
-{
-	return __deflate_decompress(src, slen, dst, dlen, ctx);
-}
-
-static struct scomp_alg scomp = {
-	.alloc_ctx		= deflate_alloc_ctx,
-	.free_ctx		= deflate_free_ctx,
-	.compress		= deflate_scompress,
-	.decompress		= deflate_sdecompress,
-	.base			= {
-		.cra_name	= "deflate",
-		.cra_driver_name = "deflate-scomp",
-		.cra_module	 = THIS_MODULE,
-	}
+	.base.cra_name		= "deflate",
+	.base.cra_driver_name	= "deflate-generic",
+	.base.cra_module	= THIS_MODULE,
 };
 
 static int __init deflate_mod_init(void)
 {
-	return crypto_register_scomp(&scomp);
+	size_t size = max(zlib_inflate_workspacesize(),
+			  zlib_deflate_workspacesize(-DEFLATE_DEF_WINBITS,
+						     DEFLATE_DEF_MEMLEVEL));
+
+	alg.reqsize = struct_size_t(struct deflate_req_ctx, workspace, size);
+	return crypto_register_acomp(&alg);
 }
 
 static void __exit deflate_mod_fini(void)
 {
-	crypto_unregister_scomp(&scomp);
+	crypto_unregister_acomp(&alg);
 }
 
 subsys_initcall(deflate_mod_init);
@@ -254,4 +162,5 @@ module_exit(deflate_mod_fini);
 MODULE_LICENSE("GPL");
 MODULE_DESCRIPTION("Deflate Compression Algorithm for IPCOMP");
 MODULE_AUTHOR("James Morris <[email protected]>");
+MODULE_AUTHOR("Ard Biesheuvel <[email protected]>");
 MODULE_ALIAS_CRYPTO("deflate");
include/crypto/scatterwalk.h +1 -1
index 32fc4473175b1d81..46dc7b21bf9ecbd0 100644
@@ -51,7 +51,7 @@ static inline struct page *scatterwalk_page(struct scatter_walk *walk)
 	return sg_page(walk->sg) + (walk->offset >> PAGE_SHIFT);
 }
 
-static inline void scatterwalk_unmap(void *vaddr)
+static inline void scatterwalk_unmap(const void *vaddr)
 {
 	kunmap_local(vaddr);
 }
Signature · 1 line
2.39.2
#38 lore raw

Re: [RFC PATCH 20/21] crypto: deflate - implement acomp API directly

On Tue, Jul 18, 2023 at 02:58:46PM +0200, Ard Biesheuvel wrote:

...

Quoted 44 lines
crypto/deflate.c +33 -9
-static int deflate_comp_init(struct deflate_ctx *ctx)
+static int deflate_process(struct acomp_req *req, struct z_stream_s *stream,
+			   int (*process)(struct z_stream_s *, int))
 {
-	int ret = 0;
-	struct z_stream_s *stream = &ctx->comp_stream;
+	unsigned int slen = req->slen;
+	unsigned int dlen = req->dlen;
+	struct scatter_walk src, dst;
+	unsigned int scur, dcur;
+	int ret;
 
-	stream->workspace = vzalloc(zlib_deflate_workspacesize(
-				-DEFLATE_DEF_WINBITS, DEFLATE_DEF_MEMLEVEL));
-	if (!stream->workspace) {
-		ret = -ENOMEM;
-		goto out;
-	}
+	stream->avail_in = stream->avail_out = 0;
+
+	scatterwalk_start(&src, req->src);
+	scatterwalk_start(&dst, req->dst);
+
+	scur = dcur = 0;
+
+	do {
+		if (stream->avail_in == 0) {
+			if (scur) {
+				slen -= scur;
+
+				scatterwalk_unmap(stream->next_in - scur);
+				scatterwalk_advance(&src, scur);
+				scatterwalk_done(&src, 0, slen);
+			}
+
+			scur = scatterwalk_clamp(&src, slen);
+			if (scur) {
+				stream->next_in = scatterwalk_map(&src);
+				stream->avail_in = scur;
+			}
+		}
+
+		if (stream->avail_out == 0) {
+			if (dcur) {

crypto/deflate.c line 73

Quoted 10 lines
crypto/deflate.c +10 -0
+				dlen -= dcur;
+
+				scatterwalk_unmap(stream->next_out - dcur);
+				scatterwalk_advance(&dst, dcur);
+				scatterwalk_done(&dst, 1, dlen);
+			}
+
+			dcur = scatterwalk_clamp(&dst, dlen);
+			if (!dcur)
+				break;

Hi Ard,

I'm unsure if this can happen. But if this break occurs in the first iteration of this do loop, then ret will be used uninitialised below.

Smatch noticed this.

Quoted 9 lines
crypto/deflate.c +9 -0
+
+			stream->next_out = scatterwalk_map(&dst);
+			stream->avail_out = dcur;
+		}
+
+		ret = process(stream, (slen == scur) ? Z_FINISH : Z_SYNC_FLUSH);
+	} while (ret == Z_OK);
+
+	if (scur)

crypto/deflate.c line 92

Quoted 10 lines
crypto/deflate.c +10 -0
+		scatterwalk_unmap(stream->next_in - scur);
+	if (dcur)
+		scatterwalk_unmap(stream->next_out - dcur);
+
+	if (ret != Z_STREAM_END)
+		return -EINVAL;
+
+	req->dlen = stream->total_out;
+	return 0;
+}

...

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

#39 lore raw

Re: [RFC PATCH 20/21] crypto: deflate - implement acomp API directly

Quoted 62 lines · Simon Horman

On Tue, Jul 18, 2023 at 02:58:46PM +0200, Ard Biesheuvel wrote:

...

Quoted 44 lines
crypto/deflate.c +33 -9
-static int deflate_comp_init(struct deflate_ctx *ctx)
+static int deflate_process(struct acomp_req *req, struct z_stream_s *stream,
+                        int (*process)(struct z_stream_s *, int))
 {
-     int ret = 0;
-     struct z_stream_s *stream = &ctx->comp_stream;
+     unsigned int slen = req->slen;
+     unsigned int dlen = req->dlen;
+     struct scatter_walk src, dst;
+     unsigned int scur, dcur;
+     int ret;
 
-     stream->workspace = vzalloc(zlib_deflate_workspacesize(
-                             -DEFLATE_DEF_WINBITS, DEFLATE_DEF_MEMLEVEL));
-     if (!stream->workspace) {
-             ret = -ENOMEM;
-             goto out;
-     }
+     stream->avail_in = stream->avail_out = 0;
+
+     scatterwalk_start(&src, req->src);
+     scatterwalk_start(&dst, req->dst);
+
+     scur = dcur = 0;
+
+     do {
+             if (stream->avail_in == 0) {
+                     if (scur) {
+                             slen -= scur;
+
+                             scatterwalk_unmap(stream->next_in - scur);
+                             scatterwalk_advance(&src, scur);
+                             scatterwalk_done(&src, 0, slen);
+                     }
+
+                     scur = scatterwalk_clamp(&src, slen);
+                     if (scur) {
+                             stream->next_in = scatterwalk_map(&src);
+                             stream->avail_in = scur;
+                     }
+             }
+
+             if (stream->avail_out == 0) {
+                     if (dcur) {

crypto/deflate.c line 73

Quoted 10 lines
crypto/deflate.c +10 -0
+                             dlen -= dcur;
+
+                             scatterwalk_unmap(stream->next_out - dcur);
+                             scatterwalk_advance(&dst, dcur);
+                             scatterwalk_done(&dst, 1, dlen);
+                     }
+
+                     dcur = scatterwalk_clamp(&dst, dlen);
+                     if (!dcur)
+                             break;

Hi Ard,

Quoted 5 lines

I'm unsure if this can happen. But if this break occurs in the first iteration of this do loop, then ret will be used uninitialised below.

Smatch noticed this.

Thanks.

This should not happen - it would mean req->dlen == 0, which is rejected before this function is even called.

Whether or not it might ever happen in practice is a different matter, of course, so I should probably initialize 'ret' to something sane.

Quoted 21 lines
Quoted 19 lines
crypto/deflate.c +19 -0
+
+                     stream->next_out = scatterwalk_map(&dst);
+                     stream->avail_out = dcur;
+             }
+
+             ret = process(stream, (slen == scur) ? Z_FINISH : Z_SYNC_FLUSH);
+     } while (ret == Z_OK);
+
+     if (scur)
+             scatterwalk_unmap(stream->next_in - scur);
+     if (dcur)
+             scatterwalk_unmap(stream->next_out - dcur);
+
+     if (ret != Z_STREAM_END)
+             return -EINVAL;
+
+     req->dlen = stream->total_out;
+     return 0;
+}

...

#40 lore raw

RFC PATCH 21/21 crypto: scompress - Drop the use of per-cpu scratch buffers

In reply to Ard Biesheuvel

The scomp to acomp adaptation layer allocates 256k of scratch buffers per CPU in order to be able to present the input provided by the caller via scatterlists as linear byte arrays to the underlying synchronous compression drivers, most of which are thin wrappers around the various compression algorithm library implementations we have in the kernel.

This sucks. With high core counts and SMT, this easily adds up to multiple megabytes that are permanently tied up for this purpose, and given that all acomp users pass either single pages or contiguous buffers in lowmem, we can optimize for this pattern and just pass the buffer directly if we can. This removes the need for scratch buffers, and along with it, the arbitrary 128k upper bound on the input and output size of the acomp API when the implementation happens to be scomp based.

So add a scomp_map_sg() helper to try and obtain the virtual addresses associated with the scatterlists, which is guaranteed to be successful 100% of the time given the existing users, which all fit the prerequisite pattern. And as a fallback for other cases, use kvmalloc with GFP_KERNEL to allocate buffers on the fly and free them again right after.

This puts the burden on future callers to either use a contiguous buffer, or deal with the potentially blocking nature of GFP_KERNEL. For IPcomp in particular, the only relevant compression algorithm is 'deflate' which is no longer implemented as an scomp, and so this change will not affect it even if we decide to convert it to take advantage of the ability to pass discontiguous scatterlists.

---
 crypto/scompress.c                  | 159 ++++++++++----------
 include/crypto/internal/scompress.h |   2 -
 2 files changed, 76 insertions(+), 85 deletions(-)
crypto/scompress.c +76 -83
index 3155cdce9116e092..1c050aa864bd604d 100644
@@ -18,24 +18,11 @@
 #include <linux/seq_file.h>
 #include <linux/slab.h>
 #include <linux/string.h>
-#include <linux/vmalloc.h>
 #include <net/netlink.h>
 
 #include "compress.h"
 
-struct scomp_scratch {
-	spinlock_t	lock;
-	void		*src;
-	void		*dst;
-};
-
-static DEFINE_PER_CPU(struct scomp_scratch, scomp_scratch) = {
-	.lock = __SPIN_LOCK_UNLOCKED(scomp_scratch.lock),
-};
-
 static const struct crypto_type crypto_scomp_type;
-static int scomp_scratch_users;
-static DEFINE_MUTEX(scomp_lock);
 
 static int __maybe_unused crypto_scomp_report(
 	struct sk_buff *skb, struct crypto_alg *alg)
@@ -58,56 +45,45 @@ static void crypto_scomp_show(struct seq_file *m, struct crypto_alg *alg)
 	seq_puts(m, "type         : scomp\n");
 }
 
-static void crypto_scomp_free_scratches(void)
-{
-	struct scomp_scratch *scratch;
-	int i;
-
-	for_each_possible_cpu(i) {
-		scratch = per_cpu_ptr(&scomp_scratch, i);
-
-		vfree(scratch->src);
-		vfree(scratch->dst);
-		scratch->src = NULL;
-		scratch->dst = NULL;
-	}
-}
-
-static int crypto_scomp_alloc_scratches(void)
-{
-	struct scomp_scratch *scratch;
-	int i;
-
-	for_each_possible_cpu(i) {
-		void *mem;
-
-		scratch = per_cpu_ptr(&scomp_scratch, i);
-
-		mem = vmalloc_node(SCOMP_SCRATCH_SIZE, cpu_to_node(i));
-		if (!mem)
-			goto error;
-		scratch->src = mem;
-		mem = vmalloc_node(SCOMP_SCRATCH_SIZE, cpu_to_node(i));
-		if (!mem)
-			goto error;
-		scratch->dst = mem;
-	}
-	return 0;
-error:
-	crypto_scomp_free_scratches();
-	return -ENOMEM;
-}
-
 static int crypto_scomp_init_tfm(struct crypto_tfm *tfm)
 {
-	int ret = 0;
+	return 0;
+}
 
-	mutex_lock(&scomp_lock);
-	if (!scomp_scratch_users++)
-		ret = crypto_scomp_alloc_scratches();
-	mutex_unlock(&scomp_lock);
+/**
+ * scomp_map_sg - Return virtual address of memory described by a scatterlist
+ *
+ * @sg:		The address of the scatterlist in memory
+ * @len:	The length of the buffer described by the scatterlist
+ *
+ * If the memory region described by scatterlist @sg consists of @len
+ * contiguous bytes in memory and is accessible via the linear mapping or via a
+ * single kmap(), return its virtual address.  Otherwise, return NULL.
+ */
+static void *scomp_map_sg(struct scatterlist *sg, unsigned int len)
+{
+	struct page *page;
+	unsigned int offset;
 
-	return ret;
+	while (sg_is_chain(sg))
+		sg = sg_next(sg);
+
+	if (!sg || sg_nents_for_len(sg, len) != 1)
+		return NULL;
+
+	page   = sg_page(sg) + (sg->offset >> PAGE_SHIFT);
+	offset = offset_in_page(sg->offset);
+
+	if (PageHighMem(page) && (offset + sg->length) > PAGE_SIZE)
+		return NULL;
+
+	return kmap_local_page(page) + offset;
+}
+
+static void scomp_unmap_sg(const void *addr)
+{
+	if (is_kmap_addr(addr))
+		kunmap_local(addr);
 }
 
 static int scomp_acomp_comp_decomp(struct acomp_req *req, int dir)
@@ -116,30 +92,52 @@ static int scomp_acomp_comp_decomp(struct acomp_req *req, int dir)
 	void **tfm_ctx = acomp_tfm_ctx(tfm);
 	struct crypto_scomp *scomp = *tfm_ctx;
 	void **ctx = acomp_request_ctx(req);
-	struct scomp_scratch *scratch;
+	void *src_alloc = NULL;
+	void *dst_alloc = NULL;
+	const u8 *src;
+	u8 *dst;
 	int ret;
 
-	if (!req->src || !req->slen || req->slen > SCOMP_SCRATCH_SIZE)
+	if (!req->src || !req->slen || !req->dst || !req->dlen)
 		return -EINVAL;
 
-	if (!req->dst || !req->dlen || req->dlen > SCOMP_SCRATCH_SIZE)
-		return -EINVAL;
-
-	scratch = raw_cpu_ptr(&scomp_scratch);
-	spin_lock(&scratch->lock);
-
-	scatterwalk_map_and_copy(scratch->src, req->src, 0, req->slen, 0);
-	if (dir)
-		ret = crypto_scomp_compress(scomp, scratch->src, req->slen,
-					    scratch->dst, &req->dlen, *ctx);
-	else
-		ret = crypto_scomp_decompress(scomp, scratch->src, req->slen,
-					      scratch->dst, &req->dlen, *ctx);
-	if (!ret) {
-		scatterwalk_map_and_copy(scratch->dst, req->dst, 0, req->dlen,
-					 1);
+	dst = scomp_map_sg(req->dst, req->dlen);
+	if (!dst) {
+		dst = dst_alloc = kvmalloc(req->dlen, GFP_KERNEL);
+		if (!dst_alloc)
+			return -ENOMEM;
 	}
-	spin_unlock(&scratch->lock);
+
+	src = scomp_map_sg(req->src, req->slen);
+	if (!src) {
+		src = src_alloc = kvmalloc(req->slen, GFP_KERNEL);
+		if (!src_alloc) {
+			ret = -ENOMEM;
+			goto out;
+		}
+		scatterwalk_map_and_copy(src_alloc, req->src, 0, req->slen, 0);
+	}
+
+	if (dir)
+		ret = crypto_scomp_compress(scomp, src, req->slen, dst,
+					    &req->dlen, *ctx);
+	else
+		ret = crypto_scomp_decompress(scomp, src, req->slen, dst,
+					      &req->dlen, *ctx);
+
+	if (src_alloc)
+		kvfree(src_alloc);
+	else
+		scomp_unmap_sg(src);
+
+	if (!ret && dst == dst_alloc)
+		scatterwalk_map_and_copy(dst, req->dst, 0, req->dlen, 1);
+out:
+	if (dst_alloc)
+		kvfree(dst_alloc);
+	else
+		scomp_unmap_sg(dst);
+
 	return ret;
 }
 
@@ -158,11 +156,6 @@ static void crypto_exit_scomp_ops_async(struct crypto_tfm *tfm)
 	struct crypto_scomp **ctx = crypto_tfm_ctx(tfm);
 
 	crypto_free_scomp(*ctx);
-
-	mutex_lock(&scomp_lock);
-	if (!--scomp_scratch_users)
-		crypto_scomp_free_scratches();
-	mutex_unlock(&scomp_lock);
 }
 
 int crypto_init_scomp_ops_async(struct crypto_tfm *tfm)
include/crypto/internal/scompress.h +0 -2
index 858fe3965ae347ef..69e593d72cbdaa99 100644
@@ -12,8 +12,6 @@
 #include <crypto/acompress.h>
 #include <crypto/algapi.h>
 
-#define SCOMP_SCRATCH_SIZE	131072
-
 struct acomp_req;
 
 struct crypto_scomp {
Signature · 1 line
2.39.2
maintainer
#41 lore raw
In reply to Ard Biesheuvel
Quoted 6 lines · Ard Biesheuvel

Patch #2 removes the support for on-the-fly allocation of destination buffers and scatterlists from the Intel QAT driver. This is never used, and not even implemented by all drivers (the HiSilicon ZIP driver does not support it). The diffstat of this patch makes a good case why the caller should be in charge of allocating the memory, not the driver.

The implementation in qat may not be optimal, but being able to allocate memory in the algorithm is a big plus for IPComp at least.

Being able to allocate memory page by page as you decompress means that:

1. We're not affected by memory fragmentation.
2. We don't waste memory by always allocating for the worst case.

Cheers,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#42 lore raw
Quoted 9 lines · Herbert Xu
Quoted 6 lines · Ard Biesheuvel

Patch #2 removes the support for on-the-fly allocation of destination buffers and scatterlists from the Intel QAT driver. This is never used, and not even implemented by all drivers (the HiSilicon ZIP driver does not support it). The diffstat of this patch makes a good case why the caller should be in charge of allocating the memory, not the driver.

Quoted 9 lines

The implementation in qat may not be optimal, but being able to allocate memory in the algorithm is a big plus for IPComp at least.

Being able to allocate memory page by page as you decompress means that:

1. We're not affected by memory fragmentation.
2. We don't waste memory by always allocating for the worst case.

So will IPcomp be able to simply assign those pages to the SKB afterwards?

maintainer
#43 lore raw
Quoted 2 lines · Ard Biesheuvel

So will IPcomp be able to simply assign those pages to the SKB afterwards?

Yes that is the idea. The network stack is very much in love with SG lists :)

Thanks,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#44 lore raw
Quoted 8 lines · Herbert Xu
Quoted 2 lines · Ard Biesheuvel

So will IPcomp be able to simply assign those pages to the SKB afterwards?

Yes that is the idea. The network stack is very much in love with SG lists :)

Fair enough. But my point remains: this requires a lot of boilerplate on the part of the driver, and it would be better if we could do this in the acomp generic layer.

Does the IPcomp case always know the decompressed size upfront?

maintainer
#45 lore raw
Quoted 4 lines · Ard Biesheuvel

Fair enough. But my point remains: this requires a lot of boilerplate on the part of the driver, and it would be better if we could do this in the acomp generic layer.

Absolutely. If the hardware can't support allocate-as-you-go then this should very much go into the generic layer.

Quoted 1 line

Does the IPcomp case always know the decompressed size upfront?

No it doesn't know. Of course, we could optimise it because we know that in 99% cases, the packet is going to be less than 4K. But we need a safety-net for those weird jumbo packets.

Thanks,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt