Skip to contentSkip to conversation

LZ4: Port LZ4 1.9.x FAST_DEC_LOOP and enable it on x86 and ARM64

PATCH New Chenxi Mao 1 participant 1 message May 17, 2019

lore
Share & tools
lore

Chenxi Mao

author
#0 lore raw

PATCH 1/1 LZ4: Port LZ4 1.9.x FAST_DEC_LOOP and enable it on x86 and ARM64

FAST_DEC_LOOP was introduced from LZ4 1.9[1].
This change would be introduce 10% on decompress operation according to LZ4 benchmark result on X86 devices. Meanwhile, LZ4 with FAST_DEC_LOOP could get improvements, however clang compiler has downgrade if FAST_DEC_LOOP enabled.

So FAST_DEC_LOOP only enabled on X86/X86-64 or ARM64 with GCC build.

LZ4 FAST_DEC_LOOP bug fixes include as well.
1. fixed read-after input in LZ4_decompress_safe() (issue 681)
2. Fix out-of-bound read in LZ4_decompress_fast() (issue 676)

PS2:
1. Move common API to lz4defs.h
2. Add PPC related inline Macro defination.
3. Leverge specific LZ4_wildCopy32 for X86 and ARM64

Here is the test result on ARM64(cortex-A53)
Benchmark via ZRAM:

Test case:
fio --bs=32k --randrepeat=1 --randseed=100 --refill_buffers \
--buffer_compress_percentage=75 \
--scramble_buffers=1 --direct=1 --loops=100 --numjobs=8 \
--filename=/dev/block/zram0 --name=seq-write --rw=write --stonewall \
--name=seq-read --rw=read --stonewall --name=seq-readwrite \
--rw=rw --stonewall --name=rand-readwrite --rw=randrw --stonewall

Patched:
READ: bw=8757MiB/s (9182MB/s)
Vanilla:
READ: bw=7765MiB/s (8142MB/s)

[1] https://github.com/lz4/lz4/releases/tag/v1.9.0

---
 lib/lz4/lz4_compress.c   |   4 +-
 lib/lz4/lz4_decompress.c | 414 ++++++++++++++++++++++++++++++++-------
 lib/lz4/lz4defs.h        |  60 +++++-
 lib/lz4/lz4hc_compress.c |   2 +-
 4 files changed, 409 insertions(+), 71 deletions(-)
lib/lz4/lz4_compress.c +2 -2
index cc7b6d4cc7c7..b703ed1ca57d 100644
@@ -322,7 +322,7 @@ static FORCE_INLINE int LZ4_compress_generic(
 				*token = (BYTE)(litLength << ML_BITS);
 
 			/* Copy Literals */
-			LZ4_wildCopy(op, anchor, op + litLength);
+			LZ4_wildCopy8(op, anchor, op + litLength);
 			op += litLength;
 		}
 
@@ -628,7 +628,7 @@ static int LZ4_compress_destSize_generic(
 				*token = (BYTE)(litLength << ML_BITS);
 
 			/* Copy Literals */
-			LZ4_wildCopy(op, anchor, op + litLength);
+			LZ4_wildCopy8(op, anchor, op + litLength);
 			op += litLength;
 		}
 
lib/lz4/lz4_decompress.c +347 -67

535 lines not on this page · raw message

lib/lz4/lz4defs.h +59 -1

84 lines not on this page · raw message

lib/lz4/lz4hc_compress.c +1 -1
index 176f03b83e56..e02e041a01d9 100644
@@ -293,7 +293,7 @@ static FORCE_INLINE int LZ4HC_encodeSequence(
 		*token = (BYTE)(length<<ML_BITS);
 
 	/* Copy Literals */
-	LZ4_wildCopy(*op, *anchor, (*op) + length);
+	LZ4_wildCopy8(*op, *anchor, (*op) + length);
 	*op += length;
 
 	/* Encode Offset */
Signature · 1 line
2.17.1