erofs: add LZMA compression support
Subsystems (3)
Gao Xiang
maintainercover erofs: add LZMA compression support
Hi folks,
Here is the EROFS LZMA support which I've been working on and off for more than half a year together with the XZ author Lasse Collin (many thanks again!).
In order to better support the LZMA fixed-sized output compression, especially for 4KiB pcluster size (which has lowest memory pressure thus useful for memory-sensitive scenarios). Lasse introduced a new LZMA header/container format called MicroLZMA [1] to minimize the original LZMA1 header (for example, we don't need to waste 4-byte dictionary size and another 8-byte uncompressed size, which can be calculated by fs directly, for each pcluster.) and enable EROFS fixed-sized output compression. Also note that MicroLZMA can be used by other things in addition to EROFS too where wasting minimal amount of space for headers is important, which can be compiled by enabling XZ_DEC_MICROLZMA.
Similar to LZ4, EROFS LZMA decompression runtime also utilizes inplace I/O and overlapped decompression to reuse the file page for compressed data temporarily with proper strategies, which is useful to ensure guaranteed performance under extremely memory pressure without extra cost.
Due to the EROFS on-disk design, LZMA algorithm can be used in the per-file basis independently and as a secondary compression algorithm besides the primary algorithm given in one file as a complement for specific access/data patterns.
The latest version can also be fetched from:git://git.kernel.org/pub/scm/linux/kernel/git/xiang/linux.git -b erofs/lzma
The latest mkfs can be fetched from:git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git -b experimental-lzma
[ The latest liblzma should be used in order to support MicroLZMA:
https://git.tukaani.org/?p=xz.git ]
Finally, there are few in-kernel fses with LZMA support but as always I might need to show some real numbers here anyway.. There are still some ongoing EROFS runtime strategies to optimize such CPU bound decompression algorithms even further. But let's do step by step:
Kernel: Linux 5.15.0-rc2
Testdata: enwik8 (100000000 bytes)
Compression level: LZMA_PRESET_DEFAULT (6)
Processor: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
SSD: INTEL SSDPEKKF360G7H (360 GB)
DDR: Samsung M471A1K43CB1-CRC (8 GB)
OS Distribution: Debian 10
Test environment:
Turbo Boost disabledscaling_governor = userspace, scaling_{min,max}_freq = 1801000
__________________________________________________
| file system | size | seq read | rand read | |_______________|___________|_ MiB/s __|__ MiB/s __| |___erofs_4k____|__41664512_|_ 59.92 __|__ 16.88 __| |___erofs_8k____|__38555648_|_ 63.86 __|__ 22.18 __| |___erofs_16k___|__36179968_|_ 63.14 __|__ 28.98 __| |___erofs_32k___|__34299904_|_ 59.14 __|__ 35.38 __| |___erofs_64k___|__32792576_|_ 57.56 __|__ 41.24 __| |__squashfs_8k__|__43274240_|_ 21.48 __|__ 15.42 __| |__squashfs_16k_|__39866368_|_ 25.10 __|__ 20.80 __| |__squashfs_64k_|__35241984_|_ 31.02 __|__ 31.36 __| |_squashfs_128k_|__33632256_|_ 36.62 __|__ 36.92 __|
Some numbers of silesia.tar:
___________________________
| file system | size | |_______________|___________| |__silesia.tar__|_211957760_| |__squashfs_8k__|__75964416_| |___erofs_4k____|__70987776_| |__squashfs_16k_|__69337088_| |___erofs_8k____|__65617920_| |___erofs_16k___|__61333504_| |__squashfs_64k_|__60669952_| |___erofs_32k___|__58331136_| |_squashfs_128k_|__58036224_| |___erofs_64k___|__56123392_|
Revised EROFS roadmap:
https://lore.kernel.org/r/20211009061150.GA7479@hsiangkao-HP-ZHAN-66-Pro-G1
[1] where the first byte of a raw LZMA stream has been replaced with a bitwise-negation of the lc/lp/pb properties byte and it has no EOPM marker since the exact compressed size is known and provided explicitly.
Hi Andrew,
Some XZ embedded (lib/xz) patches by Lasse are sent out together in this series although they're irrelevant to MicroLZMA but quite coupled. Can I send a pull request together with EROFS LZMA support for 5.16 then? Many thanks in advance!
Thanks,
Gao Xiang
Gao Xiang (2):
erofs: rename some generic methods in decompressor
erofs: lzma compression support
Lasse Collin (5):
lib/xz: Avoid overlapping memcpy() with invalid input with in-place
decompression
lib/xz: Validate the value before assigning it to an enum variable lib/xz: Move s->lzma.len = 0 initialization to lzma_reset() lib/xz: Add MicroLZMA decoder
lib/xz, lib/decompress_unxz.c: Fix spelling in comments
fs/erofs/Kconfig | 16 ++ fs/erofs/Makefile | 1 + fs/erofs/compress.h | 16 ++ fs/erofs/decompressor.c | 73 +++++---- fs/erofs/decompressor_lzma.c | 290 +++++++++++++++++++++++++++++++++++ fs/erofs/erofs_fs.h | 14 +- fs/erofs/internal.h | 22 +++ fs/erofs/super.c | 17 +- fs/erofs/zdata.c | 4 +- fs/erofs/zdata.h | 7 - fs/erofs/zmap.c | 5 +- include/linux/xz.h | 106 +++++++++++++ lib/decompress_unxz.c | 10 +- lib/xz/Kconfig | 13 ++ lib/xz/xz_dec_lzma2.c | 182 +++++++++++++++++++++- lib/xz/xz_dec_stream.c | 6 +- lib/xz/xz_dec_syms.c | 9 +- lib/xz/xz_private.h | 3 + 18 files changed, 725 insertions(+), 69 deletions(-) create mode 100644 fs/erofs/decompressor_lzma.c
Signature · 1 line
2.20.1
Gao Xiang
maintainerPATCH 1/7 lib/xz: Avoid overlapping memcpy() with invalid input with in-place decompression
From: Lasse Collin <[email protected]>
With valid files, the safety margin described in lib/decompress_unxz.c ensures that these buffers cannot overlap. But if the uncompressed size of the input is larger than the caller thought, which is possible when the input file is invalid/corrupt, the buffers can overlap. Obviously the result will then be garbage (and usually the decoder will return an error too) but no other harm will happen when such an over-run occurs.
This change only affects uncompressed LZMA2 chunks and so this should have no effect on performance.
--- lib/decompress_unxz.c | 2 +- lib/xz/xz_dec_lzma2.c | 21 +++++++++++++++++++-- 2 files changed, 20 insertions(+), 3 deletions(-)
lib/decompress_unxz.c +1 -1
@@ -167,7 +167,7 @@* memeq and memzero are not used much and any remotely sane implementation * is fast enough. memcpy/memmove speed matters in multi-call mode, but * the kernel image is decompressed in single-call mode, in which only -* memcpy speed can matter and only if there is a lot of uncompressible data +* memmove speed can matter and only if there is a lot of uncompressible data * (LZMA2 stores uncompressible chunks in uncompressed form). Thus, the * functions below should just be kept small; it's probably not worth * optimizing for speed.
lib/xz/xz_dec_lzma2.c +19 -2
@@ -387,7 +387,14 @@ static void dict_uncompressed(struct dictionary *dict, struct xz_buf *b, *left -= copy_size; -memcpy (dict->buf + dict->pos, b->in + b->in_pos, copy_size); +/* +* If doing in-place decompression in single-call mode and the +* uncompressed size of the file is larger than the caller +* thought (i.e. it is invalid input!), the buffers below may +* overlap and cause undefined behavior with memcpy(). +* With valid inputs memcpy() would be fine here. +*/ +memmove (dict->buf + dict->pos, b->in + b->in_pos, copy_size); dict->pos += copy_size;if (dict->full < dict->pos)
@@ -397,7 +404,11 @@ static void dict_uncompressed(struct dictionary *dict, struct xz_buf *b,if (dict->pos == dict->end) dict->pos =0 ; -memcpy (b->out + b->out_pos, b->in + b->in_pos, +/* +* Like above but for multi-call mode: use memmove() +* to avoid undefined behavior with invalid input. +*/ +memmove (b->out + b->out_pos, b->in + b->in_pos, copy_size); }
@@ -421,6 +432,12 @@ static uint32_t dict_flush(struct dictionary *dict, struct xz_buf *b)if (dict->pos == dict->end) dict->pos =0 ; +/* +* These buffers cannot overlap even if doing in-place +* decompression because in multi-call mode dict->buf +* has been allocated by us in this file; it's not +* provided by the caller like in single-call mode. +*/ memcpy (b->out + b->out_pos, dict->buf + dict->start, copy_size); }
Signature · 1 line
2.20.1
Gao Xiang
maintainerPATCH 2/7 lib/xz: Validate the value before assigning it to an enum variable
In reply to Gao XiangFrom: Lasse Collin <[email protected]>
This might matter, for example, if the underlying type of enum xz_check was a signed char. In such a case the validation wouldn't have caught an unsupported header. I don't know if this problem can occur in the kernel on any arch but it's still good to fix it because some people might copy the XZ code to their own projects from Linux instead of the upstream XZ Embedded repository.
This change may increase the code size by a few bytes. An alternative would have been to use an unsigned int instead of enum xz_check but using an enumeration looks cleaner.
--- lib/xz/xz_dec_stream.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
lib/xz/xz_dec_stream.c +3 -3
@@ -402,12 +402,12 @@ static enum xz_ret dec_stream_header(struct xz_dec *s)* we will accept other check types too, but then the check won't * be verified and a warning (XZ_UNSUPPORTED_CHECK) will be given. */ +if (s->temp.buf[HEADER_MAGIC_SIZE +1 ] >XZ_CHECK_MAX ) +return XZ_OPTIONS_ERROR ; + s->check_type = s->temp.buf[HEADER_MAGIC_SIZE +1 ];#ifdef XZ_DEC_ANY_CHECK -if (s->check_type >XZ_CHECK_MAX ) -return XZ_OPTIONS_ERROR ; -if (s->check_type >XZ_CHECK_CRC32 )return XZ_UNSUPPORTED_CHECK ;#else
Signature · 1 line
2.20.1
Gao Xiang
maintainerPATCH 3/7 lib/xz: Move s->lzma.len = 0 initialization to lzma_reset()
In reply to Gao XiangFrom: Lasse Collin <[email protected]>
It's a more logical place even if the resetting needs to be done only once per LZMA2 stream (if lzma_reset() called in the middle of an LZMA2 stream, .len will already be 0).
--- lib/xz/xz_dec_lzma2.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
lib/xz/xz_dec_lzma2.c +1 -2
@@ -791,6 +791,7 @@ static void lzma_reset(struct xz_dec_lzma2 *s) s->lzma.rep1 =0 ; s->lzma.rep2 =0 ; s->lzma.rep3 =0 ; + s->lzma.len =0 ;/* * All probabilities are initialized to the same value. This hack
@@ -1174,8 +1175,6 @@ XZ_EXTERN enum xz_ret xz_dec_lzma2_reset(struct xz_dec_lzma2 *s, uint8_t props) } } - s->lzma.len =0 ; - s->lzma2.sequence =SEQ_CONTROL ; s->lzma2.need_dict_reset = true;
Signature · 1 line
2.20.1
Gao Xiang
maintainerPATCH 4/7 lib/xz: Add MicroLZMA decoder
In reply to Gao XiangFrom: Lasse Collin <[email protected]>
MicroLZMA is a yet another header format variant where the first byte of a raw LZMA stream (without the end of stream marker) has been replaced with a bitwise-negation of the lc/lp/pb properties byte. MicroLZMA was created to be used in EROFS but can be used by other things too where wasting minimal amount of space for headers is important.
This is implemented using most of the LZMA2 code as is so the amount of new code is small. The API has a few extra features compared to the XZ decoder. On the other hand, the API lacks XZ_BUF_ERROR support which is important to take into account when using this API.
MicroLZMA doesn't support BCJ filters. In theory they could be added later as there are many unused/reserved values for the first byte of the compressed stream but in practice it is somewhat unlikely to happen due to a few implementation reasons.
--- include/linux/xz.h | 106 ++++++++++++++++++++++++++++ lib/xz/Kconfig | 13 ++++ lib/xz/xz_dec_lzma2.c | 156 +++++++++++++++++++++++++++++++++++++++++- lib/xz/xz_dec_syms.c | 9 ++- lib/xz/xz_private.h | 3 + 5 files changed, 284 insertions(+), 3 deletions(-)
include/linux/xz.h +106 -0
@@ -233,6 +233,112 @@ XZ_EXTERN void xz_dec_reset(struct xz_dec *s);*/ XZ_EXTERN void xz_dec_end(struct xz_dec *s); +/* +* Decompressor for MicroLZMA, an LZMA variant with a very minimal header. +* See xz_dec_microlzma_alloc() below for details. +* +* These functions aren't used or available in preboot code and thus aren't +* marked with XZ_EXTERN. This avoids warnings about static functions that +* are never defined. +*/ +/** +* struct xz_dec_microlzma - Opaque type to hold the MicroLZMA decoder state +*/ +struct xz_dec_microlzma ; + +/** +* xz_dec_microlzma_alloc() - Allocate memory for the MicroLZMA decoder +* @mode XZ_SINGLE or XZ_PREALLOC +* @dict_size LZMA dictionary size. This must be at least 4 KiB and +* at most 3 GiB. +* +* In contrast to xz_dec_init(), this function only allocates the memory +* and remembers the dictionary size. xz_dec_microlzma_reset() must be used +* before calling xz_dec_microlzma_run(). +* +* The amount of allocated memory is a little less than 30 KiB with XZ_SINGLE. +* With XZ_PREALLOC also a dictionary buffer of dict_size bytes is allocated. +* +* On success, xz_dec_microlzma_alloc() returns a pointer to +* struct xz_dec_microlzma. If memory allocation fails or +* dict_size is invalid, NULL is returned. +* +* The compressed format supported by this decoder is a raw LZMA stream +* whose first byte (always 0x00) has been replaced with bitwise-negation +* of the LZMA properties (lc/lp/pb) byte. For example, if lc/lp/pb is +* 3/0/2, the first byte is 0xA2. This way the first byte can never be 0x00. +* Just like with LZMA2, lc + lp <= 4 must be true. The LZMA end-of-stream +* marker must not be used. The unused values are reserved for future use. +* This MicroLZMA header format was created for use in EROFS but may be used +* by others too. +*/ +extern struct xz_dec_microlzma *xz_dec_microlzma_alloc (enum xz_mode mode, +uint32_t dict_size); + +/** +* xz_dec_microlzma_reset() - Reset the MicroLZMA decoder state +* @s Decoder state allocated using xz_dec_microlzma_alloc() +* @comp_size Compressed size of the input stream +* @uncomp_size Uncompressed size of the input stream. A value smaller +* than the real uncompressed size of the input stream can +* be specified if uncomp_size_is_exact is set to false. +* uncomp_size can never be set to a value larger than the +* expected real uncompressed size because it would eventually +* result in XZ_DATA_ERROR. +* @uncomp_size_is_exact This is an int instead of bool to avoid +* requiring stdbool.h. This should normally be set to true. +* When this is set to false, error detection is weaker. +*/ +extern void xz_dec_microlzma_reset (struct xz_dec_microlzma *s, +uint32_t comp_size,uint32_t uncomp_size, +int uncomp_size_is_exact); + +/** +* xz_dec_microlzma_run() - Run the MicroLZMA decoder +* @s Decoder state initialized using xz_dec_microlzma_reset() +* @b: Input and output buffers +* +* This works similarly to xz_dec_run() with a few important differences. +* Only the differences are documented here. +* +* The only possible return values are XZ_OK, XZ_STREAM_END, and +* XZ_DATA_ERROR. This function cannot return XZ_BUF_ERROR: if no progress +* is possible due to lack of input data or output space, this function will +* keep returning XZ_OK. Thus, the calling code must be written so that it +* will eventually provide input and output space matching (or exceeding) +* comp_size and uncomp_size arguments given to xz_dec_microlzma_reset(). +* If the caller cannot do this (for example, if the input file is truncated +* or otherwise corrupt), the caller must detect this error by itself to +* avoid an infinite loop. +* +* If the compressed data seems to be corrupt, XZ_DATA_ERROR is returned. +* This can happen also when incorrect dictionary, uncompressed, or +* compressed sizes have been specified. +* +* With XZ_PREALLOC only: As an extra feature, b->out may be NULL to skip over +* uncompressed data. This way the caller doesn't need to provide a temporary +* output buffer for the bytes that will be ignored. +* +* With XZ_SINGLE only: In contrast to xz_dec_run(), the return value XZ_OK +* is also possible and thus XZ_SINGLE is actually a limited multi-call mode. +* After XZ_OK the bytes decoded so far may be read from the output buffer. +* It is possible to continue decoding but the variables b->out and b->out_pos +* MUST NOT be changed by the caller. Increasing the value of b->out_size is +* allowed to make more output space available; one doesn't need to provide +* space for the whole uncompressed data on the first call. The input buffer +* may be changed normally like with XZ_PREALLOC. This way input data can be +* provided from non-contiguous memory. +*/ +extern enum xz_ret xz_dec_microlzma_run (struct xz_dec_microlzma *s, +struct xz_buf *b); + +/** +* xz_dec_microlzma_end() - Free the memory allocated for the decoder state +* @s: Decoder state allocated using xz_dec_microlzma_alloc(). +* If s is NULL, this function does nothing. +*/ +extern void xz_dec_microlzma_end (struct xz_dec_microlzma *s); +/* * Standalone build (userspace build or in-kernel build for boot time use) * needs a CRC32 implementation. For normal in-kernel use, kernel's own
lib/xz/Kconfig +13 -0
@@ -39,6 +39,19 @@ config XZ_DEC_SPARC default y select XZ_DEC_BCJ +config XZ_DEC_MICROLZMA + bool "MicroLZMA decoder" + default n + help + MicroLZMA is a header format variant where the first byte + of a raw LZMA stream (without the end of stream marker) has + been replaced with a bitwise-negation of the lc/lp/pb + properties byte. MicroLZMA was created to be used in EROFS + but can be used by other things too where wasting minimal + amount of space for headers is important. + + Unless you know that you need this, say N. + endif config XZ_DEC_BCJ
lib/xz/xz_dec_lzma2.c +154 -2
178 lines not on this page · raw message
lib/xz/xz_dec_syms.c +8 -1
@@ -15,8 +15,15 @@ EXPORT_SYMBOL(xz_dec_reset);EXPORT_SYMBOL (xz_dec_run);EXPORT_SYMBOL (xz_dec_end); +#ifdef CONFIG_XZ_DEC_MICROLZMA +EXPORT_SYMBOL (xz_dec_microlzma_alloc); +EXPORT_SYMBOL (xz_dec_microlzma_reset); +EXPORT_SYMBOL (xz_dec_microlzma_run); +EXPORT_SYMBOL (xz_dec_microlzma_end); +#endif +MODULE_DESCRIPTION ("XZ decompressor" ); -MODULE_VERSION ("1.0" ); +MODULE_VERSION ("1.1" );MODULE_AUTHOR ("Lasse Collin <[email protected]> and Igor Pavlov" );/*
lib/xz/xz_private.h +3 -0
@@ -37,6 +37,9 @@# ifdef CONFIG_XZ_DEC_SPARC # define XZ_DEC_SPARC # endif +# ifdef CONFIG_XZ_DEC_MICROLZMA +# define XZ_DEC_MICROLZMA +# endif # define memeq (a, b, size) (memcmp(a, b, size) == 0)# define memzero (buf, size) memset(buf, 0, size)# endif
Signature · 1 line
2.20.1
Gao Xiang
maintainerPATCH 5/7 lib/xz, lib/decompress_unxz.c: Fix spelling in comments
In reply to Gao XiangFrom: Lasse Collin <[email protected]>
uncompressible -> incompressible
non-splitted -> non-split
--- lib/decompress_unxz.c | 10 +++++----- lib/xz/xz_dec_lzma2.c | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-)
lib/decompress_unxz.c +5 -5
@@ -20,8 +20,8 @@* * The worst case for in-place decompression is that the beginning of * the file is compressed extremely well, and the rest of the file is -* uncompressible. Thus, we must look for worst-case expansion when the -* compressor is encoding uncompressible data. +* incompressible. Thus, we must look for worst-case expansion when the +* compressor is encoding incompressible data. * * The structure of the .xz file in case of a compressed kernel is as follows. * Sizes (as bytes) of the fields are in parenthesis.
@@ -58,7 +58,7 @@* uncompressed size of the payload is in practice never less than the * payload size itself. The LZMA2 format would allow uncompressed size * to be less than the payload size, but no sane compressor creates such -* files. LZMA2 supports storing uncompressible data in uncompressed form, +* files. LZMA2 supports storing incompressible data in uncompressed form, * so there's never a need to create payloads whose uncompressed size is * smaller than the compressed size. *
@@ -167,8 +167,8 @@* memeq and memzero are not used much and any remotely sane implementation * is fast enough. memcpy/memmove speed matters in multi-call mode, but * the kernel image is decompressed in single-call mode, in which only -* memmove speed can matter and only if there is a lot of uncompressible data -* (LZMA2 stores uncompressible chunks in uncompressed form). Thus, the +* memmove speed can matter and only if there is a lot of incompressible data +* (LZMA2 stores incompressible chunks in uncompressed form). Thus, the * functions below should just be kept small; it's probably not worth * optimizing for speed. */
lib/xz/xz_dec_lzma2.c +1 -1
@@ -520,7 +520,7 @@ static __always_inline void rc_normalize(struct rc_dec *rc)* functions so that the compiler is supposed to be able to more easily avoid * an extra branch. In this particular version of the LZMA decoder, this * doesn't seem to be a good idea (tested with GCC 3.3.6, 3.4.6, and 4.3.3 -* on x86). Using a non-splitted version results in nicer looking code too. +* on x86). Using a non-split version results in nicer looking code too. * * NOTE: This must return an int. Do not make it return a bool or the speed * of the code generated by GCC 3.x decreases 10-15 %. (GCC 4.3 doesn't care,
Signature · 1 line
2.20.1
Gao Xiang
maintainerPATCH 6/7 erofs: rename some generic methods in decompressor
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
Previously, some LZ4 methods were named with `generic'. However, while evaluating the effective LZMA approach, it seems they aren't quite generic at all (e.g. no need preparing dstpages for most LZMA cases.)
Avoid such naming instead.
--- fs/erofs/decompressor.c | 65 ++++++++++++++++++++--------------------- 1 file changed, 31 insertions(+), 34 deletions(-)
fs/erofs/decompressor.c +31 -34
@@ -17,13 +17,8 @@#endif struct z_erofs_decompressor { -/* -* if destpages have sparsed pages, fill them with bounce pages. -* it also check whether destpages indicate continuous physical memory. -*/ -int (*prepare_destpages)(struct z_erofs_decompress_req *rq, -struct list_head *pagepool); -int (*decompress)(struct z_erofs_decompress_req *rq,u8 *out); +int (*decompress)(struct z_erofs_decompress_req *rq, +struct list_head *pagepool);char *name; };
@@ -63,8 +58,12 @@ int z_erofs_load_lz4_config(struct super_block *sb,return erofs_pcpubuf_growsize (sbi->lz4.max_pclusterblks); } -static int z_erofs_lz4_prepare_destpages (struct z_erofs_decompress_req *rq, -struct list_head *pagepool) +/* +* Fill all gaps with bounce pages if it's a sparse page list. Also check if +* all physical pages are consecutive, which can be seen for moderate CR. +*/ +static int z_erofs_lz4_prepare_dstpages (struct z_erofs_decompress_req *rq, +struct list_head *pagepool) {const unsigned int nr =PAGE_ALIGN (rq->pageofs_out + rq->outputsize) >>PAGE_SHIFT ;
@@ -119,7 +118,7 @@ static int z_erofs_lz4_prepare_destpages(struct z_erofs_decompress_req *rq,return kaddr ?1 :0 ; } -static void *z_erofs_handle_inplace_io (struct z_erofs_decompress_req *rq, +static void *z_erofs_lz4_handle_inplace_io (struct z_erofs_decompress_req *rq,void *inpage,unsigned int *inputmargin,int *maptype,bool support_0padding) {
@@ -189,7 +188,8 @@ static void *z_erofs_handle_inplace_io(struct z_erofs_decompress_req *rq,return src; } -static int z_erofs_lz4_decompress (struct z_erofs_decompress_req *rq,u8 *out) +static int z_erofs_lz4_decompress_mem (struct z_erofs_decompress_req *rq, +u8 *out) {unsigned int inputmargin;u8 *headpage, *src;
@@ -216,8 +216,8 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) } rq->inputsize -= inputmargin; - src =z_erofs_handle_inplace_io (rq, headpage, &inputmargin, &maptype, - support_0padding); + src =z_erofs_lz4_handle_inplace_io (rq, headpage, &inputmargin, + &maptype, support_0padding);if (IS_ERR (src))return PTR_ERR (src);
@@ -257,17 +257,6 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out)return ret; } -static struct z_erofs_decompressor decompressors[] = { - [Z_EROFS_COMPRESSION_SHIFTED ] = { - .name ="shifted" - }, - [Z_EROFS_COMPRESSION_LZ4 ] = { - .prepare_destpages = z_erofs_lz4_prepare_destpages, - .decompress = z_erofs_lz4_decompress, - .name ="lz4" - }, -}; -static void copy_from_pcpubuf (struct page **out,const char *dst,unsigned short pageofs_out,unsigned int outputsize)
@@ -295,12 +284,11 @@ static void copy_from_pcpubuf(struct page **out, const char *dst, } } -static int z_erofs_decompress_generic (struct z_erofs_decompress_req *rq, -struct list_head *pagepool) +static int z_erofs_lz4_decompress (struct z_erofs_decompress_req *rq, +struct list_head *pagepool) {const unsigned int nrpages_out =PAGE_ALIGN (rq->pageofs_out + rq->outputsize) >>PAGE_SHIFT ; -const struct z_erofs_decompressor *alg = decompressors + rq->alg;unsigned int dst_maptype;void *dst;int ret;
@@ -325,7 +313,7 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq,return PTR_ERR (dst); rq->inplace_io = false; - ret = alg->decompress (rq, dst); + ret =z_erofs_lz4_decompress_mem (rq, dst);if (!ret)copy_from_pcpubuf (rq->out, dst, rq->pageofs_out, rq->outputsize);
@@ -336,7 +324,7 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq, }/* general decoding path which can be used for all cases */ - ret = alg->prepare_destpages (rq, pagepool); + ret =z_erofs_lz4_prepare_dstpages (rq, pagepool);if (ret <0 )return ret;if (ret) {
@@ -351,7 +339,7 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq, dst_maptype =2 ; dstmap_out: - ret = alg->decompress (rq, dst + rq->pageofs_out); + ret =z_erofs_lz4_decompress_mem (rq, dst + rq->pageofs_out);if (!dst_maptype)kunmap_atomic (dst);
@@ -360,7 +348,7 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq,return ret; } -static int z_erofs_shifted_transform (const struct z_erofs_decompress_req *rq, +static int z_erofs_shifted_transform (struct z_erofs_decompress_req *rq,struct list_head *pagepool) { const unsigned int nrpages_out =
@@ -399,10 +387,19 @@ static int z_erofs_shifted_transform(const struct z_erofs_decompress_req *rq,return 0 ; } +static struct z_erofs_decompressor decompressors[] = { + [Z_EROFS_COMPRESSION_SHIFTED ] = { + .decompress = z_erofs_shifted_transform, + .name ="shifted" + }, + [Z_EROFS_COMPRESSION_LZ4 ] = { + .decompress = z_erofs_lz4_decompress, + .name ="lz4" + }, +}; +int z_erofs_decompress (struct z_erofs_decompress_req *rq,struct list_head *pagepool) { -if (rq->alg ==Z_EROFS_COMPRESSION_SHIFTED ) -return z_erofs_shifted_transform (rq, pagepool); -return z_erofs_decompress_generic (rq, pagepool); +return decompressors[rq->alg].decompress (rq, pagepool); }
Signature · 1 line
2.20.1
Re: [PATCH 6/7] erofs: rename some generic methods in decompressor
Quoted 9 lines · Gao Xiang
From: Gao Xiang <[email protected]>
Previously, some LZ4 methods were named with `generic'. However, while evaluating the effective LZMA approach, it seems they aren't quite generic at all (e.g. no need preparing dstpages for most LZMA cases.)
Avoid such naming instead.
Signed-off-by Gao Xiang <[email protected]>
Thanks,
Gao Xiang
maintainerPATCH 7/7 erofs: lzma compression support
In reply to Gao XiangFrom: Gao Xiang <[email protected]>
Add MicroLZMA support in order to maximize compression ratios for specific scenarios. For example, it's useful for low-end embedded boards and as a secondary algorithm in a file for specific access patterns.
MicroLZMA is a new container format for raw LZMA1, which was created by Lasse Collin aiming to minimize old LZMA headers and get rid of unnecessary EOPM (end of payload marker) as well as to enable fixed-sized output compression, especially for 4KiB pclusters.
Similar to LZ4, inplace I/O approach is used to minimize runtime memory footprint when dealing with I/O. Overlapped decompression is handled with 1) bounced buffer for data under processing or 2) extra short-lived pages from the on-stack pagepool which will be shared in the same read request (128KiB for example).
--- fs/erofs/Kconfig | 16 ++ fs/erofs/Makefile | 1 + fs/erofs/compress.h | 16 ++ fs/erofs/decompressor.c | 12 +- fs/erofs/decompressor_lzma.c | 290 +++++++++++++++++++++++++++++++++++ fs/erofs/erofs_fs.h | 14 +- fs/erofs/internal.h | 22 +++ fs/erofs/super.c | 17 +- fs/erofs/zdata.c | 4 +- fs/erofs/zdata.h | 7 - fs/erofs/zmap.c | 5 +- 11 files changed, 383 insertions(+), 21 deletions(-) create mode 100644 fs/erofs/decompressor_lzma.c
fs/erofs/Kconfig +16 -0
@@ -76,3 +76,19 @@ config EROFS_FS_ZIP Enable fixed-sized output compression for EROFS. If you don't want to enable compression feature, say N. + +config EROFS_FS_ZIP_LZMA + bool "EROFS LZMA compressed data support" + depends on EROFS_FS_ZIP + select XZ_DEC + select XZ_DEC_MICROLZMA + help + Saying Y here includes support for reading EROFS file systems + containing LZMA compressed data, specifically called microLZMA. it + gives better compression ratios than the LZ4 algorithm, at the + expense of more CPU overhead. + + LZMA support is an experimental feature for now and so most file + systems will be readable without selecting this option. + + If unsure, say N.
fs/erofs/Makefile +1 -0
@@ -4,3 +4,4 @@ obj-$(CONFIG_EROFS_FS) += erofs.o erofs-objs := super.o inode.o data.o namei.o dir.o utils.o pcpubuf.o erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o erofs-$(CONFIG_EROFS_FS_ZIP) += decompressor.o zmap.o zdata.o +erofs-$(CONFIG_EROFS_FS_ZIP_LZMA) += decompressor_lzma.o
fs/erofs/compress.h +16 -0
@@ -20,6 +20,12 @@ struct z_erofs_decompress_req {bool inplace_io, partial_decoding; }; +struct z_erofs_decompressor { +int (*decompress)(struct z_erofs_decompress_req *rq, +struct list_head *pagepool); +char *name; +}; +/* some special page->private (unsigned long, see below) */ #define Z_EROFS_SHORTLIVED_PAGE (-1UL << 2)#define Z_EROFS_PREALLOCATED_PAGE (-2UL << 2)
@@ -75,7 +81,17 @@ static inline bool z_erofs_put_shortlivedpage(struct list_head *pagepool, return true; } +#define MNGD_MAPPING (sbi) ((sbi)->managed_cache->i_mapping) +static inline bool erofs_page_is_managed (const struct erofs_sb_info *sbi, +struct page *page) +{ +return page->mapping ==MNGD_MAPPING (sbi); +} +int z_erofs_decompress (struct z_erofs_decompress_req *rq,struct list_head *pagepool); +/* prototypes for specific algorithms */ +int z_erofs_lzma_decompress (struct z_erofs_decompress_req *rq, +struct list_head *pagepool);#endif
fs/erofs/decompressor.c +6 -6
@@ -16,12 +16,6 @@#define LZ4_DECOMPRESS_INPLACE_MARGIN (srcsize) (((srcsize) >> 8) + 32)#endif -struct z_erofs_decompressor { -int (*decompress)(struct z_erofs_decompress_req *rq, -struct list_head *pagepool); -char *name; -}; -int z_erofs_load_lz4_config (struct super_block *sb,struct erofs_super_block *dsb,struct z_erofs_lz4_cfgs *lz4,int size)
@@ -396,6 +390,12 @@ static struct z_erofs_decompressor decompressors[] = { .decompress = z_erofs_lz4_decompress, .name = "lz4" }, +#ifdef CONFIG_EROFS_FS_ZIP_LZMA + [Z_EROFS_COMPRESSION_LZMA] = { + .decompress = z_erofs_lzma_decompress, + .name = "lzma" + }, +#endif }; int z_erofs_decompress(struct z_erofs_decompress_req *rq,
fs/erofs/decompressor_lzma.c new file +290 -0
290 lines not on this page · raw message
fs/erofs/erofs_fs.h +12 -2
@@ -249,10 +249,11 @@ struct erofs_inode_chunk_index {/* available compression algorithm types (for h_algorithmtype) */ enum { -Z_EROFS_COMPRESSION_LZ4 =0 , +Z_EROFS_COMPRESSION_LZ4 =0 , +Z_EROFS_COMPRESSION_LZMA =1 ,Z_EROFS_COMPRESSION_MAX }; -#define Z_EROFS_ALL_COMPR_ALGS (1 << (Z_EROFS_COMPRESSION_MAX - 1)) +#define Z_EROFS_ALL_COMPR_ALGS ((1 << Z_EROFS_COMPRESSION_MAX) - 1)/* 14 bytes (+ length field = 16 bytes) */ struct z_erofs_lz4_cfgs {
@@ -261,6 +262,15 @@ struct z_erofs_lz4_cfgs {u8 reserved[10 ]; } __packed; +/* 14 bytes (+ length field = 16 bytes) */ +struct z_erofs_lzma_cfgs { +__le32 dict_size; +__le16 format; +u8 reserved[8 ]; +} __packed; + +#define Z_EROFS_LZMA_MAX_DICT_SIZE (8 * Z_EROFS_PCLUSTER_MAX_SIZE) +/* * bit 0 : COMPACTED_2B indexes (0 - off; 1 - on) * e.g. for 4k logical cluster size, 4B if compacted 2B is off;
fs/erofs/internal.h +22 -0
@@ -381,6 +381,8 @@ struct erofs_map_blocks {* approach instead if possible since it's more metadata lightweight.) */ #define EROFS_GET_BLOCKS_FIEMAP 0x0002 +/* Used to map the whole extent if non-negligible data is requested for LZMA */ +#define EROFS_GET_BLOCKS_READMORE 0x0004enum {Z_EROFS_COMPRESSION_SHIFTED =Z_EROFS_COMPRESSION_MAX ,
@@ -502,6 +504,26 @@ static inline int z_erofs_load_lz4_config(struct super_block *sb, }#endif /* !CONFIG_EROFS_FS_ZIP */ +#ifdef CONFIG_EROFS_FS_ZIP_LZMA +int z_erofs_lzma_init (void ); +void z_erofs_lzma_exit (void ); +int z_erofs_load_lzma_config (struct super_block *sb, +struct erofs_super_block *dsb, +struct z_erofs_lzma_cfgs *lzma,int size); +#else +static inline int z_erofs_lzma_init (void ) {return 0 ; } +static inline int z_erofs_lzma_exit (void ) {return 0 ; } +static inline int z_erofs_load_lzma_config (struct super_block *sb, +struct erofs_super_block *dsb, +struct z_erofs_lzma_cfgs *lzma,int size) { +if (lzma) { +erofs_err (sb,"lzma algorithm isn't enabled" ); +return -EINVAL ; + } +return 0 ; +} +#endif /* !CONFIG_EROFS_FS_ZIP */ +#define EFSCORRUPTED EUCLEAN/* Filesystem is corrupted */ #endif /* __EROFS_INTERNAL_H */
fs/erofs/super.c +14 -3
@@ -225,6 +225,9 @@ static int erofs_load_compr_cfgs(struct super_block *sb,case Z_EROFS_COMPRESSION_LZ4 : ret =z_erofs_load_lz4_config (sb, dsb, data, size);break ; +case Z_EROFS_COMPRESSION_LZMA : + ret =z_erofs_load_lzma_config (sb, dsb, data, size); +break ;default :DBG_BUGON (1 ); ret = -EFAULT ;
@@ -706,6 +709,10 @@ static int __init erofs_module_init(void)if (err) goto shrinker_err; + err =z_erofs_lzma_init (); +if (err) + goto lzma_err; +erofs_pcpubuf_init (); err =z_erofs_init_zip_subsystem ();if (err)
@@ -720,6 +727,8 @@ static int __init erofs_module_init(void) fs_err:z_erofs_exit_zip_subsystem (); zip_err: +z_erofs_lzma_exit (); +lzma_err:erofs_exit_shrinker (); shrinker_err:kmem_cache_destroy (erofs_inode_cachep);
@@ -730,11 +739,13 @@ static int __init erofs_module_init(void) static void __exit erofs_module_exit(void) {unregister_filesystem (&erofs_fs_type); -z_erofs_exit_zip_subsystem (); -erofs_exit_shrinker (); -/* Ensure all RCU free inodes are safe before cache is destroyed. */ +/* Ensure all RCU free inodes / pclusters are safe to be destroyed. */ rcu_barrier (); + +z_erofs_exit_zip_subsystem (); +z_erofs_lzma_exit (); +erofs_exit_shrinker ();kmem_cache_destroy (erofs_inode_cachep);erofs_pcpubuf_exit (); }
fs/erofs/zdata.c +2 -2
@@ -1394,8 +1394,8 @@ static void z_erofs_pcluster_readmore(struct z_erofs_decompress_frontend *f,if (backmost) { map->m_la = end; -/* TODO: pass in EROFS_GET_BLOCKS_READMORE for LZMA later */ - err =z_erofs_map_blocks_iter (inode, map,0 ); + err =z_erofs_map_blocks_iter (inode, map, +EROFS_GET_BLOCKS_READMORE );if (err)return ;
fs/erofs/zdata.h +0 -7
@@ -94,13 +94,6 @@ struct z_erofs_decompressqueue { } u; }; -#define MNGD_MAPPING (sbi) ((sbi)->managed_cache->i_mapping) -static inline bool erofs_page_is_managed (const struct erofs_sb_info *sbi, -struct page *page) -{ -return page->mapping ==MNGD_MAPPING (sbi); -} -#define Z_EROFS_ONLINEPAGE_COUNT_BITS 2#define Z_EROFS_ONLINEPAGE_COUNT_MASK ((1 << Z_EROFS_ONLINEPAGE_COUNT_BITS) - 1)#define Z_EROFS_ONLINEPAGE_INDEX_SHIFT (Z_EROFS_ONLINEPAGE_COUNT_BITS)
fs/erofs/zmap.c +4 -1
@@ -671,7 +671,10 @@ int z_erofs_map_blocks_iter(struct inode *inode,else map->m_algorithmformat = vi->z_algorithmtype[0 ]; -if (flags &EROFS_GET_BLOCKS_FIEMAP ) { +if ((flags &EROFS_GET_BLOCKS_FIEMAP ) || + ((flags &EROFS_GET_BLOCKS_READMORE ) && + map->m_algorithmformat ==Z_EROFS_COMPRESSION_LZMA && + map->m_llen >=EROFS_BLKSIZ )) { err =z_erofs_get_extent_decompressedlen (&m);if (!err) map->m_flags |=EROFS_MAP_FULL_MAPPED ;
Signature · 1 line
2.20.1
Re: [PATCH 7/7] erofs: lzma compression support
Quoted 12 lines · Gao Xiang
From: Gao Xiang <[email protected]>
Add MicroLZMA support in order to maximize compression ratios for specific scenarios. For example, it's useful for low-end embedded boards and as a secondary algorithm in a file for specific access patterns.
MicroLZMA is a new container format for raw LZMA1, which was created by Lasse Collin aiming to minimize old LZMA headers and get rid of unnecessary EOPM (end of payload marker) as well as to enable fixed-sized output compression, especially for 4KiB pclusters.
Quoted 7 lines
Similar to LZ4, inplace I/O approach is used to minimize runtime memory footprint when dealing with I/O. Overlapped decompression is handled with 1) bounced buffer for data under processing or 2) extra short-lived pages from the on-stack pagepool which will be shared in the same read request (128KiB for example).
Signed-off-by Gao Xiang <[email protected]>
Thanks,
Gao Xiang
maintainerOn Mon, Oct 11, 2021 at 05:31:38AM +0800, Gao Xiang wrote:
...
Quoted 7 lines
Hi Andrew,
Some XZ embedded (lib/xz) patches by Lasse are sent out together in this series although they're irrelevant to MicroLZMA but quite coupled. Can I send a pull request together with EROFS LZMA support for 5.16 then? Many thanks in advance!
ping.. I've tested EROFS LZMA support with my own ro_fsstress. I'd like to apply them into -next for wider integration tests even though MicroLZMA itself is relatively independent decoder (it mainly calls raw LZMA decoding functions) thus it should not impact all the current users.
Thanks,
Gao Xiang
Quoted 41 lines
Thanks,
Gao XiangGao Xiang (2):
erofs: rename some generic methods in decompressor
erofs: lzma compression supportLasse Collin (5):
lib/xz: Avoid overlappingmemcpy()with invalid input with in-place
decompression
lib/xz: Validate the value before assigning it to an enum variable lib/xz: Moves->lzma.len= 0 initialization tolzma_reset()lib/xz: Add MicroLZMA decoder
lib/xz,lib/decompress_unxz.c: Fix spelling in commentsfs/erofs/Kconfig | 16 ++ fs/erofs/Makefile | 1 + fs/erofs/compress.h | 16 ++ fs/erofs/decompressor.c | 73 +++++---- fs/erofs/decompressor_lzma.c | 290 +++++++++++++++++++++++++++++++++++ fs/erofs/erofs_fs.h | 14 +- fs/erofs/internal.h | 22 +++ fs/erofs/super.c | 17 +- fs/erofs/zdata.c | 4 +- fs/erofs/zdata.h | 7 - fs/erofs/zmap.c | 5 +- include/linux/xz.h | 106 +++++++++++++ lib/decompress_unxz.c | 10 +- lib/xz/Kconfig | 13 ++ lib/xz/xz_dec_lzma2.c | 182 +++++++++++++++++++++- lib/xz/xz_dec_stream.c | 6 +- lib/xz/xz_dec_syms.c | 9 +- lib/xz/xz_private.h | 3 + 18 files changed, 725 insertions(+), 69 deletions(-) create mode 100644 fs/erofs/decompressor_lzma.cSignature · 1 line
2.20.1