lib/lz4: explicitly support in-place decompression
PATCH 1/1 lib/lz4: explicitly support in-place decompression
commit 89b158635ad79574bde8e94d45dad33f8cf09549 upstream.
LZ4 final literal copy could be overlapped when doing in-place decompression, so it's unsafe to just use memcpy() on an optimized memcpy approach but memmove() instead.
Upstream LZ4 has updated this years ago [1] (and the impact is non-sensible [2] plus only a few bytes remain), this commit just synchronizes LZ4 upstream code to the kernel side as well.
It can be observed as EROFS in-place decompression failure on specific files when X86_FEATURE_ERMS is unsupported, memcpy() optimization of commit 59daa706fbec ("x86, mem:
Optimize memcpy by avoiding memory false dependece") will be enabled then.
Currently most modern x86-CPUs support ERMS, these CPUs just use "rep movsb" approach so no problem at all. However, it can still be verified with forcely disabling ERMS feature...
arch/x86/lib/memcpy_64.S:
ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \
- "jmp memcpy_erms", X86_FEATURE_ERMS
+ "jmp memcpy_orig", X86_FEATURE_ERMS
We didn't observe any strange on arm64/arm/x86 platform before since most memcpy() would behave in an increasing address order ("copy upwards" [3]) and it's the correct order of in-place decompression but it really needs an update to memmove() for sure considering it's an undefined behavior according to the standard and some unique optimization already exists in the kernel.
[1] https://github.com/lz4/lz4/commit/33cb8518ac385835cc17be9a770b27b40cd0e15b
[2] https://github.com/lz4/lz4/pull/717#issuecomment-497818921
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=12518
---
Hi,
Please kindly consider these two backports to 5.4.y and 5.10.y LTS kernels, and the reason shown as above (it could cause lz4 in-place decompression (mainly EROFS) failure due to the different designed memcpy overlapped behavior on x86 if ERMS is unsupported.) The lz4 upstream commit itself has been merged for 2 years. And the linux upstream commit is also merged for months without any other regression.
And in principle, it won't have any real impact at all, so I think it's now safe to backport this to LTS kernels for unsupported ERMS x86s.
Thanks,
Gao Xiang
lib/lz4/lz4_decompress.c | 6 +++++- lib/lz4/lz4defs.h | 2 ++ 2 files changed, 7 insertions(+), 1 deletion(-)
lib/lz4/lz4_decompress.c +5 -1
@@ -260,7 +260,11 @@ static FORCE_INLINE int LZ4_decompress_generic( } } -memcpy (op, ip, length); +/* +* supports overlapping memory regions; only matters +* for in-place decompression scenarios +*/ +LZ4_memmove (op, ip, length); ip += length; op += length;
lib/lz4/lz4defs.h +2 -0
@@ -137,6 +137,8 @@ static FORCE_INLINE void LZ4_writeLE16(void *memPtr, U16 value)return put_unaligned_le16 (value, memPtr); } +#define LZ4_memmove (dst, src, size) __builtin_memmove(dst, src, size) + static FORCE_INLINE void LZ4_copy8(void *dst, const void *src) {#if LZ4_ARCH64
Signature · 1 line
1.8.3.1
PATCH 1/1 lib/lz4: explicitly support in-place decompression
commit 89b158635ad79574bde8e94d45dad33f8cf09549 upstream.
LZ4 final literal copy could be overlapped when doing in-place decompression, so it's unsafe to just use memcpy() on an optimized memcpy approach but memmove() instead.
Upstream LZ4 has updated this years ago [1] (and the impact is non-sensible [2] plus only a few bytes remain), this commit just synchronizes LZ4 upstream code to the kernel side as well.
It can be observed as EROFS in-place decompression failure on specific files when X86_FEATURE_ERMS is unsupported, memcpy() optimization of commit 59daa706fbec ("x86, mem:
Optimize memcpy by avoiding memory false dependece") will be enabled then.
Currently most modern x86-CPUs support ERMS, these CPUs just use "rep movsb" approach so no problem at all. However, it can still be verified with forcely disabling ERMS feature...
arch/x86/lib/memcpy_64.S:
ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \
- "jmp memcpy_erms", X86_FEATURE_ERMS
+ "jmp memcpy_orig", X86_FEATURE_ERMS
We didn't observe any strange on arm64/arm/x86 platform before since most memcpy() would behave in an increasing address order ("copy upwards" [3]) and it's the correct order of in-place decompression but it really needs an update to memmove() for sure considering it's an undefined behavior according to the standard and some unique optimization already exists in the kernel.
[1] https://github.com/lz4/lz4/commit/33cb8518ac385835cc17be9a770b27b40cd0e15b
[2] https://github.com/lz4/lz4/pull/717#issuecomment-497818921
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=12518
---
Hi,
Please kindly consider these two backports to 5.4.y and 5.10.y LTS kernels, and the reason shown as above (it could cause lz4 in-place decompression (mainly EROFS) failure due to the different designed memcpy overlapped behavior on x86 if ERMS is unsupported.) The lz4 upstream commit itself has been merged for 2 years. And the linux upstream commit is also merged for months without any other regression.
And in principle, it won't have any real impact at all, so I think it's now safe to backport this to LTS kernels for unsupported ERMS x86s.
Thanks,
Gao Xiang
lib/lz4/lz4_decompress.c | 6 +++++- lib/lz4/lz4defs.h | 1 + 2 files changed, 6 insertions(+), 1 deletion(-)
lib/lz4/lz4_decompress.c +5 -1
@@ -263,7 +263,11 @@ static FORCE_INLINE int LZ4_decompress_generic( } } -LZ4_memcpy (op, ip, length); +/* +* supports overlapping memory regions; only matters +* for in-place decompression scenarios +*/ +LZ4_memmove (op, ip, length); ip += length; op += length;
lib/lz4/lz4defs.h +1 -0
@@ -146,6 +146,7 @@ static FORCE_INLINE void LZ4_writeLE16(void *memPtr, U16 value)* environments. This is needed when decompressing the Linux Kernel, for example. */ #define LZ4_memcpy (dst, src, size) __builtin_memcpy(dst, src, size) +#define LZ4_memmove (dst, src, size) __builtin_memmove(dst, src, size) static FORCE_INLINE void LZ4_copy8(void *dst, const void *src) {
Signature · 1 line
1.8.3.1
Greg Kroah-Hartman
maintainerQuoted 57 lines · Gao Xiang
commit
89b158635ad79574bde8e94d45dad33f8cf09549upstream.LZ4 final literal copy could be overlapped when doing in-place decompression, so it's unsafe to just use
memcpy()on an optimized memcpy approach butmemmove()instead.Upstream LZ4 has updated this years ago [1] (and the impact is non-sensible [2] plus only a few bytes remain), this commit just synchronizes LZ4 upstream code to the kernel side as well.
It can be observed as EROFS in-place decompression failure on specific files when
X86_FEATURE_ERMSis unsupported,memcpy()optimization of commit59daa706fbec("x86, mem:
Optimize memcpy by avoiding memory false dependece") will be enabled then.Currently most modern x86-CPUs support ERMS, these CPUs just use "rep movsb" approach so no problem at all. However, it can still be verified with forcely disabling ERMS feature...
arch/x86/lib/memcpy_64.S:ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \(quoted diff) +1
-1- "jmp memcpy_erms", X86_FEATURE_ERMS + "jmp memcpy_orig", X86_FEATURE_ERMSWe didn't observe any strange on arm64/arm/x86 platform before since most
memcpy()would behave in an increasing address order ("copy upwards" [3]) and it's the correct order of in-place decompression but it really needs an update tomemmove()for sure considering it's an undefined behavior according to the standard and some unique optimization already exists in the kernel.[1] https://github.com/lz4/lz4/commit/33cb8518ac385835cc17be9a770b27b40cd0e15b
[2] https://github.com/lz4/lz4/pull/717#issuecomment-497818921
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=12518Reviewed-by Nick Terrell <[email protected]>Cc Yann Collet <[email protected]>Cc Miao Xie <[email protected]>Cc Chao Yu <[email protected]>Cc Li Guifu <[email protected]>Cc Guo Xuenan <[email protected]>Signed-off-by Andrew Morton <[email protected]>Signed-off-by Linus Torvalds <[email protected]>Signed-off-by Gao Xiang <[email protected]>---
Hi,Please kindly consider these two backports to 5.4.y and 5.10.y LTS kernels, and the reason shown as above (it could cause lz4 in-place decompression (mainly EROFS) failure due to the different designed memcpy overlapped behavior on x86 if ERMS is unsupported.) The lz4 upstream commit itself has been merged for 2 years. And the linux upstream commit is also merged for months without any other regression.
Quoted 3 lines
And in principle, it won't have any real impact at all, so I think it's now safe to backport this to LTS kernels for unsupported ERMS x86s.
Both now queued up, thanks!
greg k-h