Skip to contentSkip to conversation

lib/lz4: explicitly support in-place decompression

PATCH New Gao Xiang 2 participants 3 messages Jun 1, 2021 – Jun 8, 2021 Stable branch

lore
Share & tools
lore

Gao Xiang

author
#0 lore raw

PATCH 1/1 lib/lz4: explicitly support in-place decompression

commit 89b158635ad79574bde8e94d45dad33f8cf09549 upstream.

LZ4 final literal copy could be overlapped when doing in-place decompression, so it's unsafe to just use memcpy() on an optimized memcpy approach but memmove() instead.

Upstream LZ4 has updated this years ago [1] (and the impact is non-sensible [2] plus only a few bytes remain), this commit just synchronizes LZ4 upstream code to the kernel side as well.

It can be observed as EROFS in-place decompression failure on specific files when X86_FEATURE_ERMS is unsupported, memcpy() optimization of commit 59daa706fbec ("x86, mem:
Optimize memcpy by avoiding memory false dependece") will be enabled then.

Currently most modern x86-CPUs support ERMS, these CPUs just use "rep movsb" approach so no problem at all. However, it can still be verified with forcely disabling ERMS feature...

arch/x86/lib/memcpy_64.S:

        ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \

- "jmp memcpy_erms", X86_FEATURE_ERMS
+ "jmp memcpy_orig", X86_FEATURE_ERMS

We didn't observe any strange on arm64/arm/x86 platform before since most memcpy() would behave in an increasing address order ("copy upwards" [3]) and it's the correct order of in-place decompression but it really needs an update to memmove() for sure considering it's an undefined behavior according to the standard and some unique optimization already exists in the kernel.

[1] https://github.com/lz4/lz4/commit/33cb8518ac385835cc17be9a770b27b40cd0e15b
[2] https://github.com/lz4/lz4/pull/717#issuecomment-497818921
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=12518

---
Hi,

Please kindly consider these two backports to 5.4.y and 5.10.y LTS kernels, and the reason shown as above (it could cause lz4 in-place decompression (mainly EROFS) failure due to the different designed memcpy overlapped behavior on x86 if ERMS is unsupported.) The lz4 upstream commit itself has been merged for 2 years. And the linux upstream commit is also merged for months without any other regression.

And in principle, it won't have any real impact at all, so I think it's now safe to backport this to LTS kernels for unsupported ERMS x86s.

Thanks,
Gao Xiang

 lib/lz4/lz4_decompress.c | 6 +++++-
 lib/lz4/lz4defs.h        | 2 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
lib/lz4/lz4_decompress.c +5 -1
index 0c9d3ad17e0f..4d0b59fa5550 100644
@@ -260,7 +260,11 @@ static FORCE_INLINE int LZ4_decompress_generic(
 				}
 			}
 
-			memcpy(op, ip, length);
+			/*
+			 * supports overlapping memory regions; only matters
+			 * for in-place decompression scenarios
+			 */
+			LZ4_memmove(op, ip, length);
 			ip += length;
 			op += length;
 
lib/lz4/lz4defs.h +2 -0
index 1a7fa9d9170f..369eb181d730 100644
@@ -137,6 +137,8 @@ static FORCE_INLINE void LZ4_writeLE16(void *memPtr, U16 value)
 	return put_unaligned_le16(value, memPtr);
 }
 
+#define LZ4_memmove(dst, src, size) __builtin_memmove(dst, src, size)
+
 static FORCE_INLINE void LZ4_copy8(void *dst, const void *src)
 {
 #if LZ4_ARCH64
Signature · 1 line
1.8.3.1
author
#1 lore raw

PATCH 1/1 lib/lz4: explicitly support in-place decompression

commit 89b158635ad79574bde8e94d45dad33f8cf09549 upstream.

LZ4 final literal copy could be overlapped when doing in-place decompression, so it's unsafe to just use memcpy() on an optimized memcpy approach but memmove() instead.

Upstream LZ4 has updated this years ago [1] (and the impact is non-sensible [2] plus only a few bytes remain), this commit just synchronizes LZ4 upstream code to the kernel side as well.

It can be observed as EROFS in-place decompression failure on specific files when X86_FEATURE_ERMS is unsupported, memcpy() optimization of commit 59daa706fbec ("x86, mem:
Optimize memcpy by avoiding memory false dependece") will be enabled then.

Currently most modern x86-CPUs support ERMS, these CPUs just use "rep movsb" approach so no problem at all. However, it can still be verified with forcely disabling ERMS feature...

arch/x86/lib/memcpy_64.S:

        ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \

- "jmp memcpy_erms", X86_FEATURE_ERMS
+ "jmp memcpy_orig", X86_FEATURE_ERMS

We didn't observe any strange on arm64/arm/x86 platform before since most memcpy() would behave in an increasing address order ("copy upwards" [3]) and it's the correct order of in-place decompression but it really needs an update to memmove() for sure considering it's an undefined behavior according to the standard and some unique optimization already exists in the kernel.

[1] https://github.com/lz4/lz4/commit/33cb8518ac385835cc17be9a770b27b40cd0e15b
[2] https://github.com/lz4/lz4/pull/717#issuecomment-497818921
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=12518

---
Hi,

Please kindly consider these two backports to 5.4.y and 5.10.y LTS kernels, and the reason shown as above (it could cause lz4 in-place decompression (mainly EROFS) failure due to the different designed memcpy overlapped behavior on x86 if ERMS is unsupported.) The lz4 upstream commit itself has been merged for 2 years. And the linux upstream commit is also merged for months without any other regression.

And in principle, it won't have any real impact at all, so I think it's now safe to backport this to LTS kernels for unsupported ERMS x86s.

Thanks,
Gao Xiang

 lib/lz4/lz4_decompress.c | 6 +++++-
 lib/lz4/lz4defs.h        | 1 +
 2 files changed, 6 insertions(+), 1 deletion(-)
lib/lz4/lz4_decompress.c +5 -1
index 00cb0d0b73e1..8a7724a6ce2f 100644
@@ -263,7 +263,11 @@ static FORCE_INLINE int LZ4_decompress_generic(
 				}
 			}
 
-			LZ4_memcpy(op, ip, length);
+			/*
+			 * supports overlapping memory regions; only matters
+			 * for in-place decompression scenarios
+			 */
+			LZ4_memmove(op, ip, length);
 			ip += length;
 			op += length;
 
lib/lz4/lz4defs.h +1 -0
index c91dd96ef629..673bd206aa98 100644
@@ -146,6 +146,7 @@ static FORCE_INLINE void LZ4_writeLE16(void *memPtr, U16 value)
  * environments. This is needed when decompressing the Linux Kernel, for example.
  */
 #define LZ4_memcpy(dst, src, size) __builtin_memcpy(dst, src, size)
+#define LZ4_memmove(dst, src, size) __builtin_memmove(dst, src, size)
 
 static FORCE_INLINE void LZ4_copy8(void *dst, const void *src)
 {
Signature · 1 line
1.8.3.1
#2 lore raw
Quoted 57 lines · Gao Xiang

commit 89b158635ad79574bde8e94d45dad33f8cf09549 upstream.

LZ4 final literal copy could be overlapped when doing in-place decompression, so it's unsafe to just use memcpy() on an optimized memcpy approach but memmove() instead.

Upstream LZ4 has updated this years ago [1] (and the impact is non-sensible [2] plus only a few bytes remain), this commit just synchronizes LZ4 upstream code to the kernel side as well.

It can be observed as EROFS in-place decompression failure on specific files when X86_FEATURE_ERMS is unsupported, memcpy() optimization of commit 59daa706fbec ("x86, mem:
Optimize memcpy by avoiding memory false dependece") will be enabled then.

Currently most modern x86-CPUs support ERMS, these CPUs just use "rep movsb" approach so no problem at all. However, it can still be verified with forcely disabling ERMS feature...

arch/x86/lib/memcpy_64.S:

        ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \
(quoted diff) +1 -1
-                     "jmp memcpy_erms", X86_FEATURE_ERMS
+                     "jmp memcpy_orig", X86_FEATURE_ERMS

We didn't observe any strange on arm64/arm/x86 platform before since most memcpy() would behave in an increasing address order ("copy upwards" [3]) and it's the correct order of in-place decompression but it really needs an update to memmove() for sure considering it's an undefined behavior according to the standard and some unique optimization already exists in the kernel.

[1] https://github.com/lz4/lz4/commit/33cb8518ac385835cc17be9a770b27b40cd0e15b
[2] https://github.com/lz4/lz4/pull/717#issuecomment-497818921
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=12518

---
Hi,

Please kindly consider these two backports to 5.4.y and 5.10.y LTS kernels, and the reason shown as above (it could cause lz4 in-place decompression (mainly EROFS) failure due to the different designed memcpy overlapped behavior on x86 if ERMS is unsupported.) The lz4 upstream commit itself has been merged for 2 years. And the linux upstream commit is also merged for months without any other regression.

Quoted 3 lines

And in principle, it won't have any real impact at all, so I think it's now safe to backport this to LTS kernels for unsupported ERMS x86s.

Both now queued up, thanks!

greg k-h