lib: zstd: Add cast to silence clang's -Wbitwise-instead-of-logical
PATCH 1/1 lib: zstd: Add cast to silence clang's -Wbitwise-instead-of-logical
A new warning in clang warns that there is an instance where boolean expressions are being used with bitwise operators instead of logical ones:
lib/zstd/decompress/huf_decompress.c:890:25: warning: use of bitwise '&' with boolean operands [-Wbitwise-instead-of-logical]
(BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~zstd does this frequently to help with performance, as logical operators have branches whereas bitwise ones do not.
To fix this warning in other cases, the expressions were placed on separate lines with the '&=' operator; however, this particular instance was moved away from that so that it could be surrounded by LIKELY, which is a macro for __builtin_expect(), to help with a performance regression, according to upstream zstd pull #1973.
Aside from switching to logical operators, which is likely undesirable in this instance, or disabling the warning outright, the solution is casting one of the expressions to an integer type to make it clear to clang that the author knows what they are doing. Add a cast to U32 to silence the warning. The first U32 cast is to silence an instance of -Wshorten-64-to-32 because __builtin_expect() returns long so it cannot be moved.
--- lib/zstd/decompress/huf_decompress.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/zstd/decompress/huf_decompress.c +1 -1
@@ -886,7 +886,7 @@ HUF_decompress4X2_usingDTable_internal_body(HUF_DECODE_SYMBOLX2_0 (op2, &bitD2);HUF_DECODE_SYMBOLX2_0 (op3, &bitD3);HUF_DECODE_SYMBOLX2_0 (op4, &bitD4); - endSignal = (U32 )LIKELY ( + endSignal = (U32 )LIKELY ((U32 ) (BIT_reloadDStreamFast (&bitD1) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast (&bitD2) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished)
Signature · 1 line
2.33.1.637.gf443b226ca
Nick Terrell
maintainerQuoted 29 lines · Nathan Chancellor
A new warning in clang warns that there is an instance where boolean expressions are being used with bitwise operators instead of logical ones:
lib/zstd/decompress/huf_decompress.c:890:25: warning: use of bitwise '&' with boolean operands [-Wbitwise-instead-of-logical](BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~zstd does this frequently to help with performance, as logical operators have branches whereas bitwise ones do not.
To fix this warning in other cases, the expressions were placed on separate lines with the '&=' operator; however, this particular instance was moved away from that so that it could be surrounded by LIKELY, which is a macro for
__builtin_expect(), to help with a performance regression, according to upstream zstd pull #1973.Aside from switching to logical operators, which is likely undesirable in this instance, or disabling the warning outright, the solution is casting one of the expressions to an integer type to make it clear to clang that the author knows what they are doing. Add a cast to U32 to silence the warning. The first U32 cast is to silence an instance of -Wshorten-64-to-32 because
__builtin_expect()returns long so it cannot be moved.
Quoted 4 lines
Thanks for the fix!
I’ll apply this patch to my linux-next tree and port it to upstream zstd.
Best,
Nick Terrell
Quoted 20 lines
---
lib/zstd/decompress/huf_decompress.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)lib/zstd/decompress/huf_decompress.c +1
-1@@ -886,7 +886,7 @@ HUF_decompress4X2_usingDTable_internal_body(HUF_DECODE_SYMBOLX2_0 (op2, &bitD2);HUF_DECODE_SYMBOLX2_0 (op3, &bitD3);HUF_DECODE_SYMBOLX2_0 (op4, &bitD4); - endSignal = (U32 )LIKELY ( + endSignal = (U32 )LIKELY ((U32 ) (BIT_reloadDStreamFast (&bitD1) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast (&bitD2) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished)Signature · 1 line
2.33.1.637.gf443b226ca
From: Nick Terrell
Quoted 15 lines
Sent: 26 October 2021 02:18
Quoted 7 lines · Nathan Chancellor
A new warning in clang warns that there is an instance where boolean expressions are being used with bitwise operators instead of logical ones:
lib/zstd/decompress/huf_decompress.c:890:25: warning: use of bitwise '&' with boolean operands [-Wbitwise-instead-of-logical]
Quoted 5 lines
(BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~zstd does this frequently to help with performance, as logical operators have branches whereas bitwise ones do not.
...
Quoted 2 lines
Quoted 2 lines
The first U32 cast is to silence an instance of -Wshorten-64-to-32 because
__builtin_expect()returns long so it cannot be moved.
Isn't enabling that warning completely stupid?
The casts required to silence it could easily cause more problems
- by hiding more important bugs. And seriously affect code readability.
...c
Quoted 4 lines
Quoted 4 lines
index
05570ed5f8be..5105e59ac04a100644lib/zstd/decompress/huf_decompress.c +0
-0@@ -886,7 +886,7 @@ HUF_decompress4X2_usingDTable_internal_body(
Quoted 8 lines
lib/zstd/decompress/huf_decompress.c HUF_decompress4X2_usingDTable_internal_body( · line 892
Quoted 8 lines
lib/zstd/decompress/huf_decompress.c +1
-1HUF_DECODE_SYMBOLX2_0 (op2, &bitD2);HUF_DECODE_SYMBOLX2_0 (op3, &bitD3);HUF_DECODE_SYMBOLX2_0 (op4, &bitD4); - endSignal = (U32 )LIKELY ( + endSignal = (U32 )LIKELY ((U32 ) (BIT_reloadDStreamFast (&bitD1) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast (&bitD2) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished)
Isn't that the same as:
((BIT_reload() & BIT_reload() & BIT_reload()) == BIT_DStream_unfinished)
which will generate much better code.
Especially on cpu without 'seteq' instructions.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Quoted 16 lines · David Laight
From: Nick Terrell
Quoted 15 lines
Sent: 26 October 2021 02:18
Quoted 7 lines · Nathan Chancellor
A new warning in clang warns that there is an instance where boolean expressions are being used with bitwise operators instead of logical ones:
lib/zstd/decompress/huf_decompress.c:890:25: warning: use of bitwise '&' with boolean operands [-Wbitwise-instead-of-logical]
Quoted 5 lines
(BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~zstd does this frequently to help with performance, as logical operators have branches whereas bitwise ones do not.
Quoted 7 lines
...
Quoted 2 lines
Quoted 2 lines
The first U32 cast is to silence an instance of -Wshorten-64-to-32 because
__builtin_expect()returns long so it cannot be moved.Isn't enabling that warning completely stupid?
The casts required to silence it could easily cause more problems
- by hiding more important bugs. And seriously affect code readability.
Which warning?
-Wbitwise-instead-of-logical is included in clang's -Wall and I do not think it should be disabled; this is the first instance of the warning that has been silenced with a cast.
-Wshorten-64-to-32 will never be enabled for Linux but zstd is a separate project that can be built for a variety of operating systems so that has to be considered when developing changes for the kernel because the kernel changes need to go upstream eventually if they touch core zstd code, otherwise they will just get blown away on the next import. Specifically, this warning was enabled on iOS:
https://github.com/facebook/zstd/pull/2062
Quoted 15 lines
...c
Quoted 4 lines
Quoted 4 lines
index
05570ed5f8be..5105e59ac04a100644lib/zstd/decompress/huf_decompress.c +0
-0@@ -886,7 +886,7 @@ HUF_decompress4X2_usingDTable_internal_body(Quoted 8 lines
lib/zstd/decompress/huf_decompress.c HUF_decompress4X2_usingDTable_internal_body( · line 892
Quoted 8 lines
lib/zstd/decompress/huf_decompress.c +1
-1HUF_DECODE_SYMBOLX2_0 (op2, &bitD2);HUF_DECODE_SYMBOLX2_0 (op3, &bitD3);HUF_DECODE_SYMBOLX2_0 (op4, &bitD4); - endSignal = (U32 )LIKELY ( + endSignal = (U32 )LIKELY ((U32 ) (BIT_reloadDStreamFast (&bitD1) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast (&bitD2) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished)Isn't that the same as:
Quoted 3 lines
((BIT_reload() & BIT_reload() & BIT_reload()) == BIT_DStream_unfinished)which will generate much better code.
Especially on cpu without 'seteq' instructions.
I don't think so. Feel free to double check my math.
BIT_reloadDStreamFast() can return either BIT_DStream_unfinished (0) or BIT_DStream_overflow (3). Let's say the second call returns BIT_DStream_overflow but the others return BIT_DStream_unfinished.
Current code:
(BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished)
(BIT_DStream_unfinished == BIT_DStream_unfinished) & (BIT_DStream_overflow == BIT_DStream_unfinished) & (BIT_DStream_unfinished == BIT_DStream_unfinished)
(1 & 0 & 1)
Final result: 0
Your suggestion:
(BIT_reloadDStreamFast(&bitD1) &
BIT_reloadDStreamFast(&bitD2) & BIT_reloadDStreamFast(&bitD3)) == BIT_DStream_unfinished
(BIT_DStream_unfinished &
BIT_DStream_overflow & BIT_DStream_unfinished) == BIT_DStream_unfinished
(0 & 3 & 0) == 0
(0) == 0
Final result: 1
Clang 13.0.0 and GCC 11.2.0 appear agree with me:
https://godbolt.org/z/M78s1TTEx
Cheers,
Nathan
From: Nathan Chancellor
Quoted 1 line
Sent: 26 October 2021 15:03
...
Quoted 9 lines
Quoted 3 lines
Isn't enabling that warning completely stupid?
The casts required to silence it could easily cause more problems
- by hiding more important bugs. And seriously affect code readability.Which warning?
-Wbitwise-instead-of-logical is included in clang's -Wall and I do not think it should be disabled; this is the first instance of the warning that has been silenced with a cast.
I'm not sure about that one.
I have a feeling it will generate false positives for carefully optimised code more often that it finds anything where 'short circuiting' will be a real gain.
Especially for values with are known to be either 0 or 1.
Quoted 7 lines
-Wshorten-64-to-32 will never be enabled for Linux but zstd is a separate project that can be built for a variety of operating systems so that has to be considered when developing changes for the kernel because the kernel changes need to go upstream eventually if they touch core zstd code, otherwise they will just get blown away on the next import. Specifically, this warning was enabled on iOS:
https://github.com/facebook/zstd/pull/2062
That one...
If you are going to enable it, then you need a static inline function to convert u64 to u32, not a C cast.
I'm sure that it won't be long before the compiler writes start an 'open season' on casts.
They really are more dangerous than the warnings they are trying to remove.
Quoted 19 lines
Quoted 15 lines
...c
Quoted 4 lines
Quoted 4 lines
index
05570ed5f8be..5105e59ac04a100644lib/zstd/decompress/huf_decompress.c +0
-0@@ -886,7 +886,7 @@ HUF_decompress4X2_usingDTable_internal_body(Quoted 8 lines
lib/zstd/decompress/huf_decompress.c HUF_decompress4X2_usingDTable_internal_body( · line 892
Quoted 8 lines
lib/zstd/decompress/huf_decompress.c +1
-1HUF_DECODE_SYMBOLX2_0 (op2, &bitD2);HUF_DECODE_SYMBOLX2_0 (op3, &bitD3);HUF_DECODE_SYMBOLX2_0 (op4, &bitD4); - endSignal = (U32 )LIKELY ( + endSignal = (U32 )LIKELY ((U32 ) (BIT_reloadDStreamFast (&bitD1) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast (&bitD2) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished)Isn't that the same as:
Quoted 3 lines
((BIT_reload() & BIT_reload() & BIT_reload()) == BIT_DStream_unfinished)which will generate much better code.
Especially on cpu without 'seteq' instructions.
Quoted 4 lines
I don't think so. Feel free to double check my math.
BIT_reloadDStreamFast()can return eitherBIT_DStream_unfinished(0) orBIT_DStream_overflow(3)....
Ah, I'd assumed that BIT_DStream_unfinished was non-zero. So you actually want:
endSignal = !(BIT() | BIT() | BIT());
Just kill the CaMeLs and unnecessary constants. Then the code becomes succint, easier to read/check etc.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Nick Terrell
maintainerQuoted 63 lines · David Laight
From: Nathan Chancellor
Quoted 1 line
Sent: 26 October 2021 15:03
...
Quoted 9 lines
Quoted 3 lines
Isn't enabling that warning completely stupid?
The casts required to silence it could easily cause more problems
- by hiding more important bugs. And seriously affect code readability.Which warning?
-Wbitwise-instead-of-logical is included in clang's -Wall and I do not think it should be disabled; this is the first instance of the warning that has been silenced with a cast.
I'm not sure about that one.
I have a feeling it will generate false positives for carefully optimised code more often that it finds anything where 'short circuiting' will be a real gain.
Especially for values with are known to be either 0 or 1.Quoted 7 lines
-Wshorten-64-to-32 will never be enabled for Linux but zstd is a separate project that can be built for a variety of operating systems so that has to be considered when developing changes for the kernel because the kernel changes need to go upstream eventually if they touch core zstd code, otherwise they will just get blown away on the next import. Specifically, this warning was enabled on iOS:
https://github.com/facebook/zstd/pull/2062That one...
If you are going to enable it, then you need a static inline function to convert u64 to u32, not a C cast.I'm sure that it won't be long before the compiler writes start an 'open season' on casts.
They really are more dangerous than the warnings they are trying to remove.Quoted 19 lines
Quoted 15 lines
...c
Quoted 4 lines
Quoted 4 lines
index
05570ed5f8be..5105e59ac04a100644lib/zstd/decompress/huf_decompress.c +0
-0@@ -886,7 +886,7 @@ HUF_decompress4X2_usingDTable_internal_body(Quoted 8 lines
lib/zstd/decompress/huf_decompress.c HUF_decompress4X2_usingDTable_internal_body( · line 892
Quoted 8 lines
lib/zstd/decompress/huf_decompress.c +1
-1HUF_DECODE_SYMBOLX2_0 (op2, &bitD2);HUF_DECODE_SYMBOLX2_0 (op3, &bitD3);HUF_DECODE_SYMBOLX2_0 (op4, &bitD4); - endSignal = (U32 )LIKELY ( + endSignal = (U32 )LIKELY ((U32 ) (BIT_reloadDStreamFast (&bitD1) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast (&bitD2) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished)Isn't that the same as:
Quoted 3 lines
((BIT_reload() & BIT_reload() & BIT_reload()) == BIT_DStream_unfinished)which will generate much better code.
Especially on cpu without 'seteq' instructions.Quoted 4 lines
I don't think so. Feel free to double check my math.
BIT_reloadDStreamFast()can return eitherBIT_DStream_unfinished(0) orBIT_DStream_overflow(3)....
Quoted 6 lines
Ah, I'd assumed that
BIT_DStream_unfinishedwas non-zero. So you actually want:endSignal = !(BIT() | BIT() | BIT());Just kill the CaMeLs and unnecessary constants. Then the code becomes succint, easier to read/check etc.
BIT_reloadDStreamFast() has a likely branch which returns BIT_DStream_unfinished. This construction is telling the compiler that it is allowed to re-order each call and collect the results. I don’t expect that it will translate directly to a set of and instructions, though I’d have to double check the assembly to be sure.
If you feel the code could be clearer, you’re welcome to submit a PR upstream! However, since is a hot loop, we generally favor performance over clarity to some extent, so it will have to be a perf neutral refactoring.
Best,
Nick Terrell
Quoted 6 lines
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Quoted 33 lines · Nick Terrell
Quoted 30 lines · Nathan Chancellor
A new warning in clang warns that there is an instance where boolean expressions are being used with bitwise operators instead of logical ones:
lib/zstd/decompress/huf_decompress.c:890:25: warning: use of bitwise '&' with boolean operands [-Wbitwise-instead-of-logical](BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~zstd does this frequently to help with performance, as logical operators have branches whereas bitwise ones do not.
To fix this warning in other cases, the expressions were placed on separate lines with the '&=' operator; however, this particular instance was moved away from that so that it could be surrounded by LIKELY, which is a macro for
__builtin_expect(), to help with a performance regression, according to upstream zstd pull #1973.Aside from switching to logical operators, which is likely undesirable in this instance, or disabling the warning outright, the solution is casting one of the expressions to an integer type to make it clear to clang that the author knows what they are doing. Add a cast to U32 to silence the warning. The first U32 cast is to silence an instance of -Wshorten-64-to-32 because
__builtin_expect()returns long so it cannot be moved.
Quoted 8 lines
Quoted 4 lines
Thanks for the fix!
I’ll apply this patch to my linux-next tree and port it to upstream zstd.
Applied to my linux-next tree [1] and backported upstream [2].
Thanks again for the patch!
-Nick
[1] https://github.com/terrelln/linux/tree/zstd-1.4.10
[2] https://github.com/facebook/zstd/pull/2849
Quoted 22 lines
Best,
Nick TerrellQuoted 19 lines
---
lib/zstd/decompress/huf_decompress.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)lib/zstd/decompress/huf_decompress.c +1
-1@@ -886,7 +886,7 @@ HUF_decompress4X2_usingDTable_internal_body(HUF_DECODE_SYMBOLX2_0 (op2, &bitD2);HUF_DECODE_SYMBOLX2_0 (op3, &bitD3);HUF_DECODE_SYMBOLX2_0 (op4, &bitD4); - endSignal = (U32 )LIKELY ( + endSignal = (U32 )LIKELY ((U32 ) (BIT_reloadDStreamFast (&bitD1) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast (&bitD2) == BIT_DStream_unfinished) & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished)Signature · 1 line
2.33.1.637.gf443b226ca