RFC crypto: consolidate and clean up compression APIs
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest21 patches 23 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/21 | crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"crypto/ crypto/ crypto/ | +18 | 7 replies | |
| 2/21 | crypto: qat - Drop support for allocating destination buffersdrivers/ drivers/ drivers/ +1 more | +1 | 0 replies | |
| 3/21 | crypto: acompress - Drop destination scatterlist allocation featurecrypto/ crypto/ crypto/ +1 more | +4 | 0 replies | |
| 4/21 | net: ipcomp: Migrate to acomp API from deprecated comp APIinclude/ include/ net/ +1 more | +79 | 1 reply | |
| 5/21 | ubifs: Pass worst-case buffer size to compression routinesfs/ | +2 | 5 replies | |
| 6/21 | ubifs: Avoid allocating buffer space unnecessarilyfs/ | +8 | 0 replies | |
| 7/21 | ubifs: Migrate to acomp compression APIfs/ fs/ fs/ +1 more | +90 | 1 reply | |
| 8/21 | zram: Migrate to acomp compression APIdrivers/ drivers/ drivers/ | +57 | 1 reply | |
| 9/21 | crypto: nx - Migrate to scomp APIdrivers/ drivers/ drivers/ +1 more | +57 | 0 replies | |
| 10/21 | crypto: 842 - drop obsolete 'comp' implementationcrypto/ | +1 | 0 replies | |
| 11/21 | crypto: deflate - drop obsolete 'comp' implementationcrypto/ | +1 | 0 replies | |
| 12/21 | crypto: lz4 - drop obsolete 'comp' implementationcrypto/ | +1 | 0 replies | |
| 13/21 | crypto: lz4hc - drop obsolete 'comp' implementationcrypto/ | +1 | 0 replies | |
| 14/21 | crypto: lzo-rle - drop obsolete 'comp' implementationcrypto/ | +1 | 0 replies | |
| 15/21 | crypto: lzo - drop obsolete 'comp' implementationcrypto/ | +1 | 0 replies | |
| 16/21 | crypto: zstd - drop obsolete 'comp' implementationcrypto/ | +1 | 0 replies | |
| 17/21 | crypto: cavium/zip - drop obsolete 'comp' implementationdrivers/ drivers/ drivers/ | +1 | 0 replies | |
| 18/21 | crypto: compress_null - drop obsolete 'comp' implementationcrypto/ crypto/ | +5 | 0 replies | |
| 19/21 | crypto: remove obsolete 'comp' compression APIDocumentation/ crypto/ crypto/ +6 more | +12 | 1 reply | |
| 20/21 | crypto: deflate - implement acomp API directlycrypto/ include/ | +113 | 2 replies | |
| 21/21 | crypto: scompress - Drop the use of per-cpu scratch bufferscrypto/ include/ | +76 | 0 replies |
Reviews trailers per patch
1 of 21 patches has a Reviewed-by or Acked-by.
| Patch | Zhihao Cheng |
|---|---|
| 1/21 | |
| 2/21 | |
| 3/21 | |
| 4/21 | |
| 5/21 | R-b R-b |
| 6/21 | |
| 7/21 | |
| 8/21 | |
| 9/21 | |
| 10/21 | |
| 11/21 | |
| 12/21 | |
| 13/21 | |
| 14/21 | |
| 15/21 | |
| 16/21 | |
| 17/21 | |
| 18/21 | |
| 19/21 | |
| 20/21 | |
| 21/21 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [RFC PATCH 00/21] crypto: consolidate and clean up compression APIs
This series is presented as an RFC, because I haven't quite convinced myself that the acomp API really needs both scatterlists and request objects to encapsulate the in- and output buffers, and perhaps there are more drastic simplifications that we might consider.
However, the current situation with comp, scomp and acomp APIs is definitely something that needs cleaning up, and so I implemented this series under the working assumption that we will keep the current acomp semantics wrt scatterlists and request objects.
Patch #1 drops zlib-deflate support in software, along with the test cases we have for it. This has no users and should have never been added.
Patch #2 removes the support for on-the-fly allocation of destination buffers and scatterlists from the Intel QAT driver. This is never used, and not even implemented by all drivers (the HiSilicon ZIP driver does not support it). The diffstat of this patch makes a good case why the caller should be in charge of allocating the memory, not the driver.
Patch #3 removes this on-the-fly allocation from the core acomp API.
Patch #4 does a minimal conversion of IPcomp to the acomp API.
Patch #5 and #6 are independent UBIFS fixes for things I ran into while working on patch #7.
Patch #7 converts UBIFS to the acomp API.
Patch #8 converts the zram block driver to the acomp API.
Patches #9 to #19 remove the existing 'comp' API implementations as well as the core plumbing, now that all clients of the API have been converted. (Note that pstore stopped using the 'comp' API as well, but these changes are already queued elsewhere)
Patch #20 converts the generic deflate compression driver to the acomp API, so that it can natively operate on discontiguous buffers, rather than requiring scratch buffers. This is the only IPcomp compression algorithm we actually implement in software in the kernel, and this conversion could help IPcomp if we decide to convert it further, and remove the code that 'linearizes' SKBs in order to present them to the compression API as a contiguous range.
Patch #21 converts the acomp-to-scomp adaptation layer so it no longer requires per-CPU scratch buffers. This takes advantage of the fact that all existing users of the acomp API pass contiguous memory regions, and so scratch buffers are only needed in exceptional cases, and can be allocated and deallocated on the fly. This removes the need for preallocated per-CPU scratch buffers that can easily add up to tens of megabytes on modern systems with high core counts and SMT.
These changes have been build tested and only lightly runtime tested. In particular, I haven't performed any thorough testing on the acomp conversions of IPcomp, UBIFS and ZRAM. Any hints on which respective methods and test cases to use here are highly appreciated.
Ard Biesheuvel (21):
crypto: scomp - Revert "add support for deflate rfc1950 (zlib)"
crypto: qat - Drop support for allocating destination buffers
crypto: acompress - Drop destination scatterlist allocation feature
net: ipcomp: Migrate to acomp API from deprecated comp API
ubifs: Pass worst-case buffer size to compression routines
ubifs: Avoid allocating buffer space unnecessarily
ubifs: Migrate to acomp compression API
zram: Migrate to acomp compression API
crypto: nx - Migrate to scomp API
crypto: 842 - drop obsolete 'comp' implementation
crypto: deflate - drop obsolete 'comp' implementation
crypto: lz4 - drop obsolete 'comp' implementation
crypto: lz4hc - drop obsolete 'comp' implementation
crypto: lzo-rle - drop obsolete 'comp' implementation
crypto: lzo - drop obsolete 'comp' implementation
crypto: zstd - drop obsolete 'comp' implementation
crypto: cavium/zip - drop obsolete 'comp' implementation
crypto: compress_null - drop obsolete 'comp' implementation
crypto: remove obsolete 'comp' compression API
crypto: deflate - implement acomp API directly
crypto: scompress - Drop the use of per-cpu scratch buffers
Documentation/crypto/architecture.rst | 2 - crypto/842.c | 63 +--- crypto/Makefile | 2 +- crypto/acompress.c | 6 - crypto/api.c | 4 - crypto/compress.c | 32 -- crypto/crypto_null.c | 31 +- crypto/crypto_user_base.c | 16 - crypto/crypto_user_stat.c | 4 - crypto/deflate.c | 386 ++++++-------------- crypto/lz4.c | 61 +--- crypto/lz4hc.c | 63 +--- crypto/lzo-rle.c | 60 +-- crypto/lzo.c | 60 +-- crypto/proc.c | 3 - crypto/scompress.c | 169 ++++----- crypto/testmgr.c | 184 +--------- crypto/testmgr.h | 75 ---- crypto/zstd.c | 56 +-- drivers/block/zram/zcomp.c | 67 +++- drivers/block/zram/zcomp.h | 7 +- drivers/block/zram/zram_drv.c | 12 +- drivers/crypto/cavium/zip/zip_crypto.c | 40 -- drivers/crypto/cavium/zip/zip_crypto.h | 10 - drivers/crypto/cavium/zip/zip_main.c | 50 +-- drivers/crypto/intel/qat/qat_common/qat_bl.c | 159 -------- drivers/crypto/intel/qat/qat_common/qat_bl.h | 6 - drivers/crypto/intel/qat/qat_common/qat_comp_algs.c | 86 +---- drivers/crypto/intel/qat/qat_common/qat_comp_req.h | 10 - drivers/crypto/nx/nx-842.c | 34 +- drivers/crypto/nx/nx-842.h | 14 +- drivers/crypto/nx/nx-common-powernv.c | 30 +- drivers/crypto/nx/nx-common-pseries.c | 32 +- fs/ubifs/compress.c | 61 +++- fs/ubifs/file.c | 46 +-- fs/ubifs/journal.c | 33 +- fs/ubifs/ubifs.h | 15 +- include/crypto/acompress.h | 21 +- include/crypto/internal/scompress.h | 2 - include/crypto/scatterwalk.h | 2 +- include/linux/crypto.h | 49 +-- include/net/ipcomp.h | 4 +- net/xfrm/xfrm_algo.c | 7 +- net/xfrm/xfrm_ipcomp.c | 107 ++++-- 44 files changed, 502 insertions(+), 1679 deletions(-) delete mode 100644 crypto/compress.c
Signature · 1 line
2.39.2